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This  report  discusses  the  results  of  a  performance  audit  of  the  assessment  system  used  to  evaluate  the 
capabilities  of  the  Afghanistan  National  Security  Forces  (ANSF).  It  includes  ten  recommendations  for 
ISAF  Joint  Command  (IJC)  and  three  for  NATO  Training  Mission-Afghanistan/Combined  Security 
Transition  Command-Afghanistan  (NTM-A/CSTC-A)  intended  to  improve  the  reliability  of  future  ANSF 
assessments  and  to  support  U.S.  and  international  efforts  to  develop  ANSF  capabilities. 

A  summary  of  this  report  is  on  page  ii.  This  performance  audit  was  conducted  by  the  Office  of  the 
Special  Inspector  General  for  Afghanistan  Reconstruction  under  the  authority  of  Public  Law  No.  110-181 
and  the  Inspector  General  Act  of  1978,  as  amended.  When  preparing  the  final  report,  we  considered 
comments  from  IJC,  NTM-A/CSTC-A,  and  the  Office  of  the  Secretary  of  Defense  (OSD).  Copies  of  these 
comments  are  included  in  appendices  II,  III,  IV,  and  V  of  this  report. 
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SIGAR  Audit-10-11  June  2010 

Actions  Needed  to  Improve  the 
Reliability  of  Afghan  Security  Force 
Assessments 

What  SIGAR  Reviewed 

The  development  of  self-sufficient  Afghan  National  Security  Forces  (ANSF)  capable  of  independently  providing 
internal  and  external  security  for  Afghanistan  is  a  key  goal  of  the  U.S.  strategy  for  Afghanistan.  For  the  last  5  years, 
the  Capability  Milestone  (CM)  rating  system  has  been  the  primary  metric  for  measuring  the  development  of  ANSF 
capabilities  against  end-state  goals.  In  late  2009,  responsibility  for  the  management  of  ANSF  capability 
assessments  moved  from  NTM-A/CSTC-A  to  IJC.  This  review— the  first  independent  evaluation  of  the  CM  rating 
system  since  its  creation  in  2005— assesses  (1)  the  reliability  of  the  Capability  Milestone  rating  system;  (2) 
summary  reports  provided  to  decision-makers;  (3)  challenges  to  U.S.  and  Coalition  assessment  efforts;  and  (4) 
systemic  ANSF  deficiencies  that  have  undermined  efforts  to  develop  unit-level  capabilities.  To  accomplish  these 
objectives,  we  conducted  interviews  with  responsible  U.S.,  Coalition,  and  Afghan  officials;  reviewed  ANSF 
assessment  documents  from  IJC  and  NTM-A/CSTC-A;  and  visited  18  Afghan  army  and  police  units.  We  conducted 
our  work  in  Washington,  D.C.,  and  Afghanistan  from  October  2009  to  May  2010  in  accordance  with  generally 
accepted  government  auditing  standards. 

What  SIGAR  Found 

As  the  United  States  and  international  community  aim  to  transfer  security  responsibility  to  Afghan  security  forces, 
having  a  reliable  system  for  monitoring  progress  toward  that  objective  is  critical.  The  CM  rating  system  has  not 
provided  reliable  or  consistent  assessments  of  ANSF  capabilities.  For  example: 

•  Measurements  used  in  the  assessment  system  have  overstated  operational  capabilities. 

•  Top-rated  ANSF  units  have  not  indicated  a  capability  to  sustain  independent  operations. 

•  The  CM  rating  system  has  inadvertently  created  disincentives  for  ANSF  development. 

•  ANSF  capability  reports  have  included  outdated  assessment  data. 

In  addition  to  these  weaknesses,  serious  challenges  affect  U.S.  and  Coalition  assessment  efforts,  including  security 
conditions,  mentor  shortages,  and  inadequate  training.  Further,  systemic  ANSF  deficiencies  have  undermined 
efforts  to  develop  unit  capabilities.  These  include  logistics  problems,  personnel  attrition,  inadequate  personnel 
authorizations,  infrastructure  deficiencies,  corruption,  drug  abuse,  and  illiteracy.  In  late  April  2010,  IJC  replaced  the 
CM  rating  system  with  a  new  unit-level  assessment  called  the  Commander's  Unit  Assessment  Tool  (CUAT).  SIGAR 
has  not  yet  assessed  the  CUAT  system;  however,  the  principles  and  recommendations  in  this  report  remain 
relevant  to  whatever  system  is  in  place  for  evaluating  the  operational  effectiveness  of  fielded  ANSF  units. 

What  SIGAR  Recommends 

To  improve  the  reliability  of  ANSF  capability  assessments,  SIGAR  is  making  ten  recommendations  to  IJC  to  improve 
the  measurement,  validation,  and  reporting  of  assessment  results;  to  increase  visibility  into  Afghan  police 
capabilities;  and  to  counteract  perverse  incentives  that  have  accompanied  the  use  of  ANSF  assessment  systems. 
SIGAR  is  also  making  three  recommendations  to  NTM-A/CSTC-A  to  improve  compliance  with  ministerial  logistics 
systems,  help  offset  the  effects  of  personnel  attrition,  and  improve  operational  effectiveness  by  extending  driver 
training. 

IJC  concurred  or  partially  concurred  with  all  ten  recommendations.  NTM-A/CSTC-A  concurred  with  all  three 
recommendations. 

For  more  information  contact:  SIGAR  Public  affairs  at  (703)  602-8742  or  PublicAffairs(a)siaar.mil 


SIGAR 


Special  Inspector  General  for  Afghanistan  Reconstruction 
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Actions  Needed  to  Improve  the  Reliability  of  Afghan  Security  Force  Assessments 


The  development  of  self-sufficient  Afghan  National  Security  Forces  (ANSF)  capable  of  independently 
providing  internal  and  external  security  for  Afghanistan  is  a  key  goal  of  the  U.S.  strategy  for  Afghanistan. 
In  the  long  term,  a  reliable  system  for  monitoring  progress  toward  that  objective  is  critical,  as  the  United 
States  and  international  community  aim  to  transfer  security  responsibility  to  Afghan  security  forces.  In 
the  short  term,  the  development  of  ANSF  capabilities  is  essential  to  the  U.S.  and  Coalition 
counterinsurgency  strategy,  which  calls  for  securing  key  population  areas  and  relies  on  bolstering  the 
Afghan  people's  confidence  in  their  government.  For  5  years,  the  Capability  Milestone  (CM)  rating 
system  has  been  the  primary  system  for  measuring  the  development  of  ANSF  capabilities  against  end- 
state  goals.  In  late  2009,  responsibility  for  the  management  of  ANSF  capability  assessments  moved  from 
NTM-A/CSTC-A  to  IJC.  The  Department  of  Defense  (DOD)  has  regularly  reported  CM  rating  summaries 
for  the  Afghan  army  and  police  to  the  President  of  the  United  States  and  Congress. 

This  report  assesses  (1)  the  reliability  of  the  CM  rating  system,  (2)  summary  reports  provided  to 
decision-makers,  (3)  challenges  to  U.S.  and  Coalition  ANSF  assessment  efforts,  and  (4)  systemic  ANSF 
deficiencies  that  have  undermined  efforts  to  develop  unit-level  capabilities. 

To  accomplish  these  objectives,  we  conducted  interviews  with  responsible  U.S.,  Coalition,  and  Afghan 
officials;  reviewed  ANSF  assessment  documents  from  ISAF  Joint  Command  (IJC)  and  NATO  Training 
Mission-Afghanistan/Combined  Security  Transition  Command-Afghanistan  (NTM-A/CSTC-A);  and  visited 
18  Afghan  army  and  police  units  located  in  Regional  Commands  Capital,  North,  and  East,  where  we 
interviewed  mentors,  partners,  and  Afghan  officials.  We  conducted  our  work  in  Washington,  D.C.,  and 
Afghanistan  from  October  2009  to  May  2010  in  accordance  with  generally  accepted  government 
auditing  standards.  A  discussion  of  our  scope  and  methodology  is  included  in  appendix  I. 


SIGAR  Audit-10-11  Security/ANSF  Capability  Ratings 


Page  1 


BACKGROUND 


The  U.S.  strategy  in  Afghanistan  depends  on  building  that  country's  capacity  to  provide  for  its  own 
security  by  training  and  equipping  the  Afghan  National  Army  (ANA)  and  Afghan  National  Police  (ANP), 
collectively  referred  to  as  the  Afghan  National  Security  Forces  (ANSF).  Since  2002,  the  United  States  has 
provided  over  $27  billion— over  half  of  all  reconstruction  dollars— for  ANSF  training,  equipping,  and 
sustainment.  Current  ANSF  force-generation  goals,  which  were  agreed  to  by  the  Afghan  Government 
and  the  international  community,  call  for  171,600  ANA  and  134,000  ANP  by  October  2011. 

Starting  in  2005,  the  United  States  used  the  CM  rating  system  to  monitor  and  report  on  progress  in 
developing  fielded  Afghan  security  forces,  which  consist  of  the  Afghanistan  National  Army  and 
Afghanistan  National  Police.  The  CM  assessment  system  grades  Afghan  army  and  police  unit  capabilities 
on  a  4-point  scale  in  which  a  score  of  1  is  awarded  to  the  most  capable  units  and  4  to  the  least.1  CM 
rating  definitions  generally  designate  CM1  units  as  capable  of  conducting  their  primary  mission  with 
limited  Coalition  support.  That  definition  aligns  with  U.S.  and  international  strategic  goals;  and  in  official 
statements,  DOD  and  ISAF  have  said  that,  at  the  CM1  level,  an  Afghan  army  or  police  unit  is  expected  to 
be  capable  of  "independent  operations." 

As  of  March  29,  2010,  IJC  reported  CM  ratings  for  150  Afghan  army  and  130  Afghan  police  units.  As 
shown  in  Table  1  and  2,  approximately  23  percent  of  those  army  units  and  12  percent  of  police  units 
received  the  highest  rating,  CM1. 

Table  1:  IJC's  Afghan  Army  CM  Ratings  as  of  March  29,  2010 


Source:  SIGAR  analysis  of  IJC  data. 

Table  2:  IJC's  Afghan  Police  CM  Ratings  as  of  March  29,  2010 


Source:  SIGAR  analysis  of  IJC  data. 

Assessment  methods  for  evaluating  army  and  police  units  have  differed.  U.S.  and  Coalition  military 
mentors  and  partners  working  with  ANSF  units  have  had  responsibility  for  performing  monthly  CM 
rating  assessments  at  the  army  kandak  (battalion)  level  and  the  police  district  or  precinct  level.2 
Mentors  guide  and  advise  ANSF  units.  Partners  do  the  same,  but  also  share  an  area  of  responsibility 


1  The  CM  system  has  also  been  used  by  NTM-A/CSTC-A  to  rate  the  capacity  of  the  Ministries  of  Defense  and 
Interior;  however,  this  report  focuses  exclusively  on  the  assessment  of  fielded  forces. 

2  As  of  April  2010,  IJC  moved  to  a  6-week  assessment  cycle. 
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with  their  assigned  ANSF  units,  frequently  co-locating  and  jointly  training,  planning,  and  executing 
operations.3 

Under  the  CM  rating  system,  police  mentors  and  partners  use  a  structured  evaluation  form  called  the 
Capability  Endorsement  (CE)  checklist.  CM  ratings  for  Afghan  police  units  depended  on  assessment  of 
the  following  factors: 

•  personnel  (including  payroll  and  personnel  systems), 

•  equipping  (including  maintenance,  supply  and  property  accountability  systems), 

•  facilities, 

•  training,  and 

•  proficiencies  in  security  sector  functions  (including  command  and  control,  intelligence,  and  force 
protection). 

By  contrast,  army  mentors  and  partners  conducted  CM  assessments  using  a  less  structured,  more 
subjective  approach  submitted  in  various  formats.  CM  ratings  for  Afghan  army  units  were  based  on 
assessments  of  the  following  factors: 

•  personnel, 

•  command  and  control, 

•  training, 

•  sustainment, 

•  equipment  on-hand, 

•  equipment  readiness,  and 

•  an  overall  assessment  measure. 

For  both  police  and  army  units,  after  monthly  assessment  reports  were  completed  at  the  unit  level,  they 
were  gathered  at  the  regional  command  level;  results  were  then  passed  up  to  the  ISAF  Joint  Command 
(IJC).4  At  IJC,  the  ANSF  Development  Assistance  Bureau  (ADAB)  had  responsibility  for  performing  quality 
control  checks  and  for  aggregating  assessment  data  into  a  comprehensive  briefing  called  the  Transition 
Readiness  Assessment  Tool  (TRAT).  The  TRAT  included  a  final  summary  of  CM  results  for  each 
assessment  cycle.  (See  figure  1  for  CM  assessment  procedures.) 

During  the  course  of  our  review,  changes  were  made  to  reshape  the  ANSF  development  program, 
including  a  realignment  of  ISAF's  command  and  control  structure.  To  achieve  greater  unity  of  command 
and  improve  operational  effectiveness,  General  McChrystal,  Commander  of  ISAF,  established  IJC  in 
October  2009  and  NTM-A  in  November  2009.  IJC  achieved  full  operational  capability  in  November  2009, 
while  NTM-A  did  the  same  in  February  2010.  As  of  April  2010,  DOD  reported  that  IJC  Fleadquarters  was 
manned  at  82.5  percent  and  NTM-A/CSTC-A  at  52  percent. 


3  Mentoring  and  partnering  typically  occurs  after  training,  once  Afghan  army  and  police  units  have  been  fielded. 

4  The  CM  rating  system  was  designed  to  be  used  at  the  army  kandak  (battalion)  or  police  district/precinct  level.  For 
example,  CM  assessments  of  the  abilities  of  army  brigades  or  corps  to  operate  effectively  as  a  whole  were  not 
performed. 
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Figure  1:  CM  Assessment  Procedures,  as  of  March  2010 


Source:  SIGAR  analysis  of  IJC  and  NTM-A/CSTC-A  information. 


In  late  March  2010,  IJC  told  us  that  plans  were  underway  to  replace  the  CM  rating  system  completely. 
Since  its  creation  in  late  2009,  IJC  has  had  responsibility  for  the  management  of  U.S.  and  Coalition 
mentors  and  partners  for  fielded  ANSF  forces,  including  the  assessment  of  ANSF  capabilities.  As  part  of 
that  responsibility,  IJC  introduced  a  new  process  by  which  Regional  Commands  report  on  governance, 
development,  and  security  for  priority  areas  of  Afghanistan.  As  of  February  2010,  the  new  process  still 
used  CM  rating  results  for  measurements  of  unit-level  ANSF  capabilities.  In  late  April  2010,  IJC  ordered 
the  use  of  a  new  unit-level  assessment  system  intended  to  replace  the  CM  rating  system  called  the 
Commander's  Unit  Assessment  Tool  (CUAT). 
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CAPABILITY  MILESTONE  RATINGS  HAVE  NOT  PROVIDED  RELIABLE  ASSESSMENTS  OF  ANSF 
EFFECTIVENESS 


CM  ratings  have  not  provided  consistent  and  reliable  measures  of  progress  toward  the  goal  of 
developing  self-sustaining  security  forces  for  Afghanistan.  Methods  of  measurement  and  validation 
used  to  generate  ANSF  capability  ratings  contributed  to  a  lack  of  equivalence  in  CM  rating  results,  in 
some  cases  obscuring  accurate  measures  of  operational  effectiveness. 

The  CM  Rating  System  Has  Yielded  Inconsistent  Measures  of  ANSF  Capabilities  and  Has 
Created  Disincentives  for  ANSF  Improvement 

The  inability  of  top-rated  units  to  sustain  their  capabilities  over  time  calls  into  question  the  effectiveness 
of  the  CM  rating  system  as  a  reliable  measure  of  ANSF  development  progress.  Furthermore,  the  CM 
rating  system  created  unintended  disincentives  for  ANSF  units  to  attain  top-rated  capabilities.  Finally, 
similarly  rated  ANSF  units  have  not  always  manifested  similar  capabilities. 

Top  CM  Ratings  May  Not  Indicate  the  Capability  to  Maintain  Self-Sustaining  Operations 

Although  the  United  States  and  ISAF  have  stated  that  the  attainment  of  "self-sustaining  operations"  is  a 
goal  for  ANSF  development,  units  obtaining  the  top  rating  in  the  CM  system,  CM1,  may  not  have  the 
capacity  to  sustain  the  gains  they  have  made.  Our  analysis  of  CM-rating  histories  showed  significant 
levels  of  regression,  or  backsliding,  in  the  capability  levels  of  fielded  army  and  police  units.  We 
evaluated  monthly  ANSF  CM  ratings  for  a  12  month  period  (February  2009  through  January  2010)  and 
found  that  38  percent  of  army  units  (52  of  137  units)  and  66  percent  of  police  districts  (67  of  101 
districts)  had  regressed  at  least  one  level  during  that  period.  When  we  considered  only  those  units  that 
had  received  the  top  rating,  CM1,  we  found  that  39  percent  of  top-rated  army  units  (20  of  51  kandaks) 
and  71  percent  of  top-rated  police  districts  (22  of  31)  had  regressed  at  least  one  CM  rating  level.  Of 
those  units  that  lost  their  CM1  status,  11  army  units  (55  percent)  and  4  police  units  (18  percent)  had 
regained  their  CM1  status  by  the  January  2010  rating  cycle. 

Police  units,  in  particular,  have  had  difficulty  sustaining  top-level  capabilities.  Officials  from  IJC,  NTM- 
A/CSTC-A,  State's  Bureau  of  International  Narcotics  and  Law  Enforcement  Affairs,  and  DynCorp 
expressed  concern  about  this  trend,  stating  that  CMl-rated  police  units  require  significant  post-CMl 
mentoring  to  sustain  training  results  and  stave  off  regression.  Mentors  and  partners  at  police  districts 
we  visited  reported  similar  concerns.  For  example,  in  February  2010,  a  Regional  Command  (RC)  North 
mentor  team,  responding  to  written  questions  we  provided,  stated,  "The  ANP  will  simply  stop  doing 
what  we  asked  them  to  do  as  soon  as  we  leave  the  area.  This  is  especially  troublesome  in  areas  of 
security  and  patrolling."  Police  mentor  and  partner  team  assessment  reports  corroborated  these 
concerns.  For  example,  an  October  2009  assessment  report  describing  a  CMl-rated  police  unit  in  Orgun, 
Paktika  Province,  stated  that  "The  District  has  well  trained  ANP  soldiers  but  this  training  needs  constant 
reinforcement." 

Police  districts  have  experienced  regressions  in  their  assessed  capabilities,  due  to  several  factors, 
including  attrition  of  personnel,  reassignments,  and  a  lack  of  sustained  mentoring  and  partnering. 

•  Attrition:  Heavy  personnel  losses  have  been  a  primary  cause  of  capability  regression  in  police 
units.  Mentors,  partners,  and  IJC  officials  stated  that  training  has  not  been  systematically 
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provided  to  new  recruits  brought  in  to  replenish  units,  causing  a  gradual  loss  of  trained 
personnel. 

•  Reassignments:  Police  mentors,  partners,  and  Afghan  commanders  stated  and  assessment 
reports  confirmed  that  reassignments  of  police  personnel— often  to  provincial  police 
headquarters— were  commonplace  and  a  significant  factor  in  reducing  police  units' 
effectiveness. 

•  Lack  of  sustained  mentoring/partnering:  A  lack  of  sustained  mentoring/partnering— a  critical 
element  in  supporting  the  development  of  police  units— has  contributed  to  the  regression  of 
some  units.  IJC  and  NTM-A/CSTC-A  officials  we  spoke  to  cited  NATO  restrictions  on  mentors, 
mentor  shortages,  and  conflicting  battle  space  priorities  as  having  contributed  to  this  pattern. 


The  CM  rating  system  itself  has  created  disincentives  for  ANSF  units  to  make  progress  toward  a  CM1 
rating  and,  thereafter,  to  become  independent  of  Coalition  support.  Mentor/partner  support  for  ANA 
units  at  the  top  rating  level  has  been  minimized  to  re-prioritize  support  for  lower-rated  units;  in  the  case 
of  ANP  units,  it  is  withdrawn  entirely.  Not  surprisingly,  ANSF  units  dislike  the  prospect  of  losing  U.S.  and 
Coalition  mentors  and  partners  who  bring  with  them  force  protection,  expertise,  supplies,  funding,  and 
prestige.  According  to  ANSF  mentors  and  partners  and  IJC  officials,  the  potential  loss  of  U.S.  and 
Coalition  support  has  been  a  direct  disincentive  to  improvement  on  the  part  of  Afghan  units.  In  addition, 
IJC's  Validation  Transition  Team  (VTT),  which  provides  independent  validation  of  CM1  capabilities  for  the 
Afghan  army,  warned  against  decreasing  the  size  of  mentor  teams  immediately  following  validation.  The 
team  reported  that  CM1  units  generally  had  residual  deficiencies  that  a  reduced  mentor  team  was  not 
equipped  to  address,  and  that  Afghan  army  units  feared  that  attaining  a  CM1  rating  would  mean  "you're 
on  your  own."  For  police  units,  full  withdrawal  of  all  mentors/partners  has  been  programmatically 
intended,  once  a  unit  has  held  a  CM1  rating  through  a  period  of  monitoring.  As  of  March  2010,  only  one 
police  district— Baghlan-e  Jadid  in  Baghlan  Province— had  graduated  to  that  stage,  with  unfortunate 
results  (see  "Overstated  Police  Capabilities:  The  Case  of  Baghlan-e  Jadid,"  page  13).  In  February  2010, 

IJC  Assessments  officials  stated  that  they  recognized  the  CM  system  had  inadvertently  provided 
disincentives  to  development. 


ANSF  Units  with  the  Same  CM  Ratings  May  Not  Have  Equivalent  Capabilities 

CM  ratings  for  ANSF  units  have  provided  relative,  not  absolute,  measures  of  unit-level  capabilities. 

Based  on  interviews,  observations,  and  reviews  of  assessment  reports,  we  found  that  relative 
differences  in  units  receiving  different  CM  ratings  were  readily  apparent.  Specifically,  units  rated  CM1 
were  generally  more  capable  than  those  rated  CM2,  and  so  forth.  However,  we  also  found  that  similarly- 
rated  units  in  different  regions  were  not  always  comparable  so  that  one  could  not  assume  that  units 
with  similar  ratings  would  have  similar  capabilities.  In  particular,  Afghan  army  units  receiving  similar 
ratings  in  different  parts  of  the  country  exhibited  disparities  in  actual  capability.  Variations  were 
pronounced  between  army  units  being  partnered  compared  with  those  mentored.  We  observed  that 
the  more  intensive  nature  of  partnering  resulted  in  increased  developmental  benefits  to  Afghan  units. 
Additionally,  planning  and  executing  joint  operations  through  partnering  allowed  U.S.  and  Coalition 
forces  to  assess  firsthand  the  effectiveness  of  Afghan  army  units  during  operations.  Mentoring,  by 
contrast,  resulted  in  a  different  type  of  exposure,  and  NATO  mentors  were  often  restricted  in  their 
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development  and  assessment  activities  due  to  national  caveats  and  guidelines  that  limited  the  scope  of 
their  activities.* * * 5 

The  Measurement  and  Validation  of  ANSF  Capabilities  May  Have  Resulted  in  Overstatement 
and  Variability  in  Assessment  Results 

Measures  used  to  determine  CM  ratings  resulted  in  overstated  operational  capabilities  and  a  lack  of 
consistency  in  assessment  results.  Inconsistencies  resulted  from  an  emphasis  on  quantitative  measures, 
biases  in  personnel  inputs,  variable  rating  definitions,  weaknesses  in  assessment  tools,  and  a  lack  of 
independent  validation  of  police  capabilities. 

Emphasis  on  Quantitative  Measures  Has  Sometimes  Obscured  the  Assessment  of  Operational 

Effectiveness 

CM  assessments  were  biased  by  a  dependence  on  quantitative  inputs  that  in  some  cases  obscured 
reliable  assessments  of  operational  effectiveness.  Quantitative  measures  of  readiness— such  as  levels  of 
manning  and  equipping— can  provide  important  gauges  of  progress  toward  force  generation  goals. 
However,  CM  ratings  depended  more  heavily  on  the  measurement  of  supplies  and  personnel  available 
to  a  unit  than  on  subjective  evaluations  of  the  unit's  ability  to  perform  its  mission  effectively.6  As  a 
result,  CM  ratings  did  not  consistently  provide  reliable  assessments  of  ANSF  readiness.  For  example,  our 
discussions  with  army  and  police  mentors  and  partners  revealed  that  in  some  fully  supplied  units  poor 
leadership,  corruption,  and  other  factors  had  significantly  diminished  operational  effectiveness. 
Conversely,  some  units  lacking  all  authorized  supplies  demonstrated  successful  operational  abilities. 

Police  CM  ratings  were  based  on  results  from  a  structured  questionnaire,  called  the  Capability 
Endorsement  (CE)  checklist  system.  According  to  police  mentors  and  partners  we  interviewed,  CM 
ratings  for  some  units  had  been  inflated  by  the  CE  form's  overdependence  on  raw  numbers  of  personnel 
and  equipment  on  hand.  As  a  result,  CM  ratings  resulting  from  use  of  the  CE  Checklist  were  not  always 
reliable  in  indicating  a  unit's  capability  level.7  In  February  2010,  a  police  mentor  team  leader  in  Kunduz 
Province,  RC-North,  responding  to  our  questions,  wrote,  "The  CM  report  spits  out  a  rating  that  to  me 
indicates  a  much  higher  level  of  competence  than  I  am  trying  to  report...  The  CM  Rating  does  not 
provide  an  accurate  assessment  of  the  district.  If  your  numbers  are  good,  the  form  will  spit  out  a  high 
CM  rating." 


National  caveats  are  limits  imposed  on  the  operations  of  international  partners'  troops,  to  ensure  those  troops 

are  used  in  a  manner  consistent  with  their  national  laws  and  policies.  DOD  has  noted  that  the  resulting  limits  on 

the  operations  of  international  partner  forces  may  limit  the  ISAF  Commander's  ability  to  utilize  his  forces  (see 

DOD's  April  28,  2010  Report  on  Progress  Toward  Security  and  Stability  in  Afghanistan,  p.17). 

6Seventy  percent  of  the  measures  listed  on  the  CE  Checklist,  which  until  the  end  of  2009  was  exclusively  used  for 
ANP  assessments,  are  quantitative  measures  of  manning,  training,  and  equipping.  The  balance  is  an  aggregate 
measure  of  answers  to  156  YES/NO/NOT  OBSERVED  questions.  Assessments  of  ANA  units  also  have  relied  heavily 
on  quantitative  measures,  which  constituted  50  percent  of  the  factors  used  in  determining  a  CM  rating. 

7Commanders  had  the  authority  to  upgrade  or  downgrade  a  unit  after  the  generation  of  a  CM  rating  using  the  CE 
Checklist. 
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Most  CM  assessments  for  Afghan  army  units  were  not  created  using  the  CE  Checklist;  instead,  they  were 
conducted  using  a  variety  of  methods  that  had  the  potential  to  include  more  subjectivity.  As  a  result, 
ANA  ratings  may  not  have  been  subject  to  the  same  reliance  on  quantitative  measures.  Nevertheless, 
our  analysis  of  ANA  TRAT  data  for  March  2010  showed  that  of  35  CMl-rated  units,  6  (over  17  percent) 
had  the  lowest  possible  "Equipment  on  Hand"  rating,  CM4,  corresponding  to  a  level  of  under  50  percent 
of  equipment  authorized.  Furthermore,  a  NATO  mentor  who  had  worked  with  Afghan  army  units  in  RC- 
South  stated  that  "the  CM  rating  system  as  it  was  used  in  RC  South  in  2008-2009  was  not  a  reliable 
predictor  of  Afghan  army  unit  effectiveness.  It  had  no  relation  to  an  Afghan  unit's  ability  to  operate 
independently." 

One  example  of  the  effect  of  the  CM  system's  reliance  on  quantitative  measures  emerged  during  our 
field  observations  of  Afghan  army  and  police  units.  We  found  that  army  and  police  units,  including 
those  rated  at  the  highest  capability  level,  CM1,  were  fully  supplied  with  vehicles  but  lacked  a  sufficient 
number  of  trained  drivers.8  For  example,  Bati  Kot,  a  top-rated  police  district  in  Nangarhar  Province,  had 
10  vehicles  on  hand,  but  only  3  capable  drivers,  at  the  time  of  our  visit  in  March  2010.  CM  assessments 
had  taken  into  account  only  the  number  of  vehicles  on  hand,  but  not  the  number  of  drivers  available  to 
operate  the  vehicles. 

Officials  at  IJC  and  NTM-A/CSTC-A  agreed  that  the  CM  system's  reliance  on  quantitative  measures  had 
sometimes  created  misleading  results.  IJC  officials  stated  that  they  expected  upcoming  changes  to  the 
ANSF  assessment  system  and  TRAT  process  to  help  remedy  the  problem  by  emphasizing  qualitative 
aspects  of  assessment. 

Personnel  Measurements  Have  Overstated  Operational  Capabilities 

Measurements  of  personnel  numbers  have  been  key  quantitative  inputs  used  to  generate  CM  ratings  for 
ANSF  units.  Although  personnel  levels  have  been  measured  differently  for  Afghan  army  and  police 
ratings,  both  methods  have  resulted  in  overstatements  of  unit  capabilities.  For  the  army,  personnel 
measurements  constituted  one  of  six  input  categories  used  to  rate  the  capabilities  of  Afghan  army  units. 
However,  personnel  inputs  used  in  ANA  unit  capability  assessments  did  not  reflect  operational  realities, 
as  they  were  based  on  measurements  of  the  number  of  personnel  assigned  to  each  unit,  without  also 
considering  the  actual  number  of  soldiers  present  for  duty.  Due  to  substantial  absences  without  leave 
(AWOL)  and  approved  absence  rates,  this  method  of  evaluation  has  falsely  inflated  measures  of 
personnel  readiness,  contributing  to  an  overstatement  of  ANA  capabilities.9 

The  effect  of  substantial  AWOL  rates  and  approved  absences  has  been  significant.  According  to  an  IJC 
report,  as  of  March  2010,  only  74  percent  of  assigned  personnel  were  present  for  duty  in  ANA-fielded 
combat  units.  The  ANA's  manning  shortage  was  confirmed  in  statements  by  VTT  officials  who  reported 


8  German  mentors  for  the  Afghan  army's  2nd  Brigade,  209th  Corps  in  Kunduz  Province  stated  that  a  lack  of  capable 
drivers  was  one  of  the  most  significant  impediments  to  the  development  of  operational  effectiveness. 

9  MOD'S  definition  of  AWOL  includes  those  personnel  absent  from  regular  duty  after  24  hours.  MOD  does  not 
consider  personnel  late  back  from  leave  to  be  AWOL.  Additionally,  according  to  officials  at  NTM-A/CSTC-A,  an 
MOD  AWOL  amnesty  was  in  place  for  ANA  personnel  through  the  end  of  the  last  solar  year,  1388.  That  amnesty 
allowed  troops  to  return  to  the  rolls  without  punishment. 
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that  "foxhole  strength"  is  often  low,  potentially  reducing  kandak  (battalions)  to  company  size.10  Our 
analysis  of  CM  rating  records  showed  that,  as  of  March  2010,  99  of  150  ANA  units  (66  percent)  received 
a  top  rating  for  personnel.  However,  even  with  a  top  rating  for  personnel,  a  unit  may  have  a  low 
number  of  personnel  present  for  duty.  For  example,  as  of  March  2010,  the  1st  kandak  of  the  3rd  Brigade, 
205th  Corps,  was  granted  a  CM1  rating  for  personnel,  based  on  a  fill  rate  of  99  percent.  Specifically,  this 
kandak  had  796  personnel  authorized  and  795  assigned.  However,  the  same  unit  had  only  469 
personnel  present  for  duty,  giving  it  a  present-for-duty  rate  of  59  percent,  which  would  have 
corresponded  to  a  CM3  personnel  rating.*  11 

Police  unit  capabilities  also  have  been  overstated  by  personnel  measurements,  which  are  one  of  several 
quantitative  inputs  used  to  generate  CM  ratings.  Personnel  inputs  for  police  unit  assessments  have 
incorporated  measures  of  authorized,  assigned,  and  available  personnel  levels.  However,  the  number  of 
"available"  personnel  does  not  correspond  with  those  present  for  duty  because  it  includes  personnel 
detailed  to  other  locations.  According  to  statements  by  police  mentor  and  partner  teams  and  our 
review  of  police  unit  assessment  records,  reassignment  has  been  a  significant  problem  at  the  district 
level,  leaving  numerous  police  units  without  sufficient  personnel  to  operate  effectively.  This  challenge 
has  been  exacerbated  by  heavy  requirements  for  manning  static  checkpoints  and  outposts,  leaving  very 
few  police  available  for  patrols  and  other  mobile  operations. 

Data  on  police  personnel,  which  have  not  been  validated,  may  not  be  reliable,  according  to  police 
mentors  and  partners.  They  reported  that  there  was  "no  accountability"  for  Afghan  personnel,  including 
no  way  to  verify  personnel  reports.  Within  each  police  district,  personnel  are  often  posted  at 
widespread  locations,  making  personnel  monitoring  particularly  challenging  in  non-permissive  security 
conditions.  Officials  from  NTM-A/CSTC-A,  IJC,  Dyncorp,  State/INL,  as  well  as  police  mentors  in  RC- 
Capital,  RC-North  and  RC-East,  stated  that  ongoing  monthly  reporting  of  police  personnel  levels  were 
not  reliable.  However,  NTM-A/CSTC-A  also  reported  that  Ministry  of  Interior  (MOI),  with  U.S.  and 
Coalition  support,  had  made  progress  in  developing  a  snapshot  inventory  of  police  through  an  ongoing 
personnel  inventory. 

CM  Rating  Level  Definitions  Lacked  Consistency 

CM  rating  definitions  have  varied,  inhibiting  a  common  understanding  of  rating  levels  among  ANSF 
mentors  and  partners  conducting  unit  assessments.  Rating  definitions  have  changed  as  the  assessment 
system  evolved;  however,  we  also  identified  contemporaneous  differences  in  CM  rating  definitions.  For 
instance,  an  IJC  FRAGO  (fragmentary  order)  dated  November  30,  2009,  regarding  TRAT  reporting 
contained  full  CM  rating  definitions  for  ANA  ratings  and  references  to  CE  form  version  5.15,  which 
contained  a  different  set  of  definitions.12  Furthermore,  an  ANA  unit  assessment  report,  completed 
shortly  after  the  IJC  FRAGO  was  issued,  contained  another  definition  of  CM1  (see  Table  3).  ANSF 
mentors  and  partners  reported  that  they  noticed  a  lack  of  consistency  in  CM  level  definitions  and  were 
concerned  that  this  would  result  in  a  lack  of  equivalence  between  assessments. 


10  The  ANA  is  transitioning  from  three  to  four  line  companies  per  kandak. 

11  Present-for-duty  as  a  rate  constitutes  the  ratio  of  present-for-duty  personnel  to  assigned  personnel. 

12ISAF  Joint  Command,  Transition  Readiness  Assessment  Tool  Reporting,  HQ  ISAF//FRAGO/42-2009,  November  30, 
2009. 
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Table  3:  Varying  Definitions  for  CM1 


Source 

Date 

Document 

CM1  Definition 

IJC 

Nov.  30,  2009 

FRAGO  042-2009 

"CM  1  unit  is  fully  capable  of  planning,  executing,  and 
sustaining  counterinsurgency  operations  at  Battalion 

Level  with  coalition  support  required  in  Close  Air  Support 
(CAS)  /  Medical  Evacuation  (MEDEVAC)  /  Indirect  (ID) 

Fires  only." 

NTM-A/CSTC-A 

Nov.  30,  2009 

Capability 

Endorsement 

checklist  form 

version  5.15 

"CM  1.  (Full  Operational  Capability  -  FOC)  The  unit, 
agency,  staff  function,  or  installation  is  capable  of 
conducting  primary  operational  mission(s).  Capability,  in 
terms  of  DOTMLPF,  is  defined  at  >85%."13 

Regional 
Command  West 

Jan.  10,  2010 

ANA  kandak 

assessment 

submission 

"Level  1  unit  is  fully  capable  of  planning,  executing,  and 
sustaining  independent  counterinsurgency  operations. 
Battalion  Level." 

Source:  SIGAR  analysis  of  data  provided  by  IJC  and  NTM-A/CSTC-A. 


Weaknesses  in  ANSF  Evaluation  Tools  Yielded  Inconsistent  Assessment  Results 


According  to  ANSF  mentors/partners,  the  CE  checklist  has  weaknesses  that  have  resulted  in  uncertainty 
among  assessors  and  inconsistent  assessment  results.  Mentors  and  partners  stated  that  they  often  did 
not  have  a  clear  understanding  of  what  was  intended  by  each  CE  assessment  question  and  were 
concerned  that  this  may  have  affected  assessment  results.  They  also  noted  that  high  levels  of  non¬ 
response  within  the  assessment  tool  did  not  prevent  the  generation  of  a  CM  rating  level  and  were  not 
noted  in  summary  reports. 

On  November  30,  2009,  IJC  ordered  all  army  and  police  mentors  and  partners  to  use  the  same 
assessment  evaluation  form:  CE  checklist  version  5.15.  Before  that  date,  ANA  mentors/partners  used  a 
less  structured  system  of  reporting,  including  more  subjective  narrative  assessments.  Even  though  the 
CE  checklist,  which  was  originally  designed  for  use  by  police  mentors/partners,  was  revised  for  use  in 
ANA  assessments,  a  number  of  questions  inappropriate  for  ANA  evaluation  remained.  For  example,  the 
form  still  stated  "Is  the  unit  aligned  with  the  most  current  Mol  approved  payroll  system  and  processes?" 
and  "Does  the  ANP  follow  Afghan  criminal  law  and  crime  procedure  requirements  for  processing 
crime?"  As  of  February  2010,  according  to  IJC  officials,  very  few  ANA  mentors/partners  had  adopted  the 
CE  checklist  for  use  in  assessing  army  units. 

Other  sources  of  variation  in  measurement  methods  have  contributed  to  concerns  about  equivalence 
between  ANSF  evaluations.  Due  to  the  less-structured  requirements  associated  with  the  original  ANA 
assessment  process  (prior  to  December  2009),  army  mentors  and  partners  provided  a  considerable 


13U.S.  military  doctrine  divides  force  development  into  domains:  doctrine,  organization,  training,  materiel, 
leadership  and  education,  personnel  and  facilities  (DOTMLPF). 
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range  of  assessment  submissions.14  Mentors  and  partners  conducting  ANP  evaluations  were  using,  at 
some  points  in  time,  entirely  different  versions  of  the  CE  checklist  that  assessed  capabilities  differently.15 
In  addition,  some  ANP  assessment  submissions  included  valuable,  subjective  narrative  evaluations 
written  by  mentors  and  partners;  however  those  were  not  required  elements  of  an  evaluation  and  were 
not  consistently  completed.  Further,  in  our  review  of  CE  checklist  submissions  for  ANP  assessments,  we 
noticed  a  wide  range  in  the  number  of  unobserved  items  on  each  CE  assessment  form.  Officials  at  NTM- 
A/CSTC-A  stated  that,  since  the  CE  checklist's  introduction  in  2005,  no  attempt  had  been  made  to 
evaluate  its  internal  consistency  or  other  aspects  of  its  effectiveness. 

Police  Units  Have  Not  Been  Independently  Validated 

VTTs  independently  validate  ANA  units  before  their  designation  as  a  CMl-rated  unit.  Applying  uniform 
capability  standards  through  an  independent  validation  was  intended  to  help  standardize  the 
assessment  process  and  to  capture  lessons  learned  regarding  ANA  units  and  their  development.  ANP 
units,  by  contrast,  have  not  been  subject  to  independent  validations.  As  of  February  2010,  4  of  7  ANA- 
dedicated  VTT  teams  had  been  reassigned  to  support  the  ANSF  training  mission.  Thus,  it  was  unclear 
whether  sufficient  resources  would  be  available  to  extend  the  program  to  the  ANP. 


14  ANA  assessment  submissions  we  reviewed  from  mentors  and  partners  included  various  formats,  such  as 
narratives  in  Word  documents,  Mission  Essential  Task  List  evaluations  in  Excel  spreadsheets,  and  PowerPoint  slide 
compilations. 

15  For  example,  unlike  version  5.x  of  the  CE  checklist,  version  4.x  did  not  assess  ANP  facilities. 
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SUMMARY  REPORTS  INCLUDED  OUTDATED  DATA  AND  OVERSTATED  ANSF  CAPABILITIES 

Decision  makers  hoping  to  gauge  progress  in  U.S.  and  Coalition  efforts  to  develop  capable  Afghan 
security  forces  depend  on  summary  reports  of  CM  rating  results.  However,  these  reports  have  included 
outdated  assessment  data  without  indicating  them  as  such.  A  lack  of  quality  control  measures  and 
resources  at  IJC  also  contributed  to  weaknesses  in  the  analysis  and  reporting  of  ANSF  assessment 
results. 

ANSF  Capability  Rating  Reports  Included  Outdated  Assessment  Data 

TRAT  reports  of  ANSF  capability  ratings  regularly  included  outdated  assessment  data  due  to  missing 
submissions  from  ANSF  mentors  and  partners,  according  to  officials  at  IJC,  NTM-A/CSTC-A,  and 
State/INL.  Officials  at  IJC  and  NTM-A/CSTC-A  stated  that  monthly  CM  assessment  submissions  were 
typically  missing  due  to:  (1)  a  lack  of  mentors/partners  to  provide  reporting;  (2)  turnover  from  one 
mentor/partner  team  to  another;  (3)  technical  problems  in  submitting  reports  from  certain  areas  of  the 
country;  and  (4)  the  reluctance  of  NATO  police  mentor  teams  to 
perform  mentoring  tasks,  due  to  security  concerns  and  national 
caveats. 


In  our  analysis  of  IJC  records  for  the  ANP,  we  found  that,  from 
September  2009  through  February  2010,  as  many  as  44  percent 
of  police  district  reports  had  been  missing  in  a  single  month  (see 
Table  4).  In  some  cases,  we  found  ratings  that  had  been  carried 
over  for  as  long  as  4  months.  We  were  unable  to  perform  a 
parallel  analysis  for  ANA  assessment  submissions  during  this 
time  period,  as  IJC  was  not  able  to  provide  a  full  set  of  ANA 
assessment  records. 


IJC  Performed  Limited  Quality  Control  over  Assessment 
Data  before  Submitting  Summary  Reports 

Although  IJC/ADAB  had  copies  of  all  submitted  ANP  assessment  files,  they  did  not  have  ANA 
assessments  on  file.  This  lack  of  access  to  the  underlying  assessment  reports  used  to  create  TRAT 
summaries  meant  that  IJC  could  perform  only  limited  quality  control  and  analysis  of  the  assessment 
data.  When  we  requested  copies  of  all  ANA  unit  assessments  for  October  2009  and  January  2010, 
IJC/ADAB  attempted  to  retrieve  copies  from  the  Regional  Commands,  but  was  unable  to  obtain  a  full  set 
of  files. 

IJC  performed  limited  quality  control  or  analysis  of  ANP  assessment  results.  According  to  officials  at 
IJC/ADAB  responsible  for  aggregating  ANP  assessment  files,  the  complex  data  included  in  the 
assessments  had  raised  concerns  because  it  appeared  to  be  unduly  consistent  from  1  month  to  the  next. 
However,  IJC/ADAB  did  not  conduct  any  analysis  of  or  quality  control  over  the  data  within  assessment 
files,  due  to  a  lack  of  resources. 


Table  4:  Missing  Police  Assessments, 
September  2009  to  February  2010 


Missing  Police 
Assessments 

SEP  09 

12  of  63  (19%) 

OCT  09 

33  of  108  (30%) 

NOV  09 

23  of  108  (21%) 

DEC  09 

32  of  72  (44%) 

JAN  10 

7  of  89  (8%) 

FEB  10 

23  of  126(18%) 
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Overstated  Police  Capabilities:  The  Case  of  Baghlan-e  Jadid 

IJC's  March  2010  TRAT  report  included  a  CM1 
rating  for  the  Baghlan-e  Jadid  police  district  in  the 
northern  Afghan  province  of  Baghlan.  The  district 
first  reached  a  CM1  rating  in  August  2008  and, 
according  to  IJC,  maintained  that  rating  for  9 
months  until  it  graduated  from  the  Focused 
Police  District  Development  (FPDD)  program  in 
June  2009.  However,  the  Baghlan-e  Jadid  police 
received  no  direct  capability  assessments  after 
that  period.  According  to  NTM-A/CSTC-A  officials, 
full  graduation  signified  that  a  police  unit  had 
reached  a  level  of  capability  at  which  it  was  "able 
to  function  on  its  own"  and  would  no  longer  be 
rated  and  monitored  by  U.S.  or  Coalition  forces. 

At  graduation,  all  U.S.  and  Coalition 
mentors/partners  were  removed.  As  of  March 
2010,  direct  assessments  of  the  police  in  that  district  had  still  not  been  conducted. 

Officials  at  IJC  and  NTM-A/CSTC-A  and  State  Department  civilian  police  advisers  and  contractors  all 
stated  that,  in  the  absence  of  any  direct  assessment,  they  had  grave  concerns  about  assuming  that 
Baghlan-e  Jadid  had  sustained  its  top-level  capabilities.  Explaining  their  doubts,  officials  cited 
increasingly  challenging  security  conditions  and  the  widespread  tendency  for  other  police  districts  to 
regress  due  to  the  lack  of  continuous  mentoring/partnering  and  heavy  attrition.  An  NTM-A/CSTC-A 
official  stated  that  two  other  ANP  districts  had  progressed  to  the  stage  just  before  graduation,  but 
regressed  after  their  original  police  mentor  teams  were  withdrawn.  IJC  and  NTM-A/CSTC-A  officials 
agreed  that  reporting  on  districts  not  receiving  monitoring  in  parallel  with  actively  evaluated  districts 
was  misleading. 

In  February  2010,  our  team  requested  support  for  a  visit  to  the  Baghlan-e  Jadid  police  district.  U.S. 
police  mentors  working  in  Regional  Command  North  stated  that  they  could  not  support  our  request 
because  the  police  district  was  "not  secure."  We  also  consulted  IJC  officials  who  said  that  the  district 
was  "overrun  with  insurgents."  One  IJC  official  commented,  "Unfortunately,  [FPDD]  graduation  has 
become  synonymous  with  CM1  (a  super  CM1  if  you  will)  that  doesn't  exist  at  all  in  reality."  He  added 
that  in  his  opinion  the  Baghlan-e  Jadid  police  force  had  "withered  away  to  the  point  that  it  barely 
functions."  Another  U.S.  military  official,  operating  from  within  RC-North  said,  "I  doubt  CM1.  Most  of 
their  police  officers  do  not  even  have  uniforms,  nor  has  the  majority  received  basic  training,  either." 

In  February  2010,  IJC  officials  told  us  that  Baghlan-e  Jadid  would  be  part  of  a  new  focus  of  intensive 
work  to  improve  certain  priority  districts  around  the  country. 


BAGHLAN 
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DOD  Reported  Capability  Ratings  for  Afghan  Police  Units  That  Had  Not  Been  Assessed  by 
Mentors  or  Partners 

Afghan  police  TRAT  summary  reports  we  reviewed  only  included  CM  ratings  for  police  units  that  had 
been  assessed  by  mentors  or  partners.  However,  in  some  reports  to  Congress,  DOD  reported  capability 
ratings  for  more  police  units  than  had  actually  been  assessed.  Units  that  had  not  been  assessed  were 
reported  by  DOD  at  the  CM-4  level.  In  biannual  reports  to  Congress,  prior  to  April  2010,  Defense 
reported  capability  ratings  for  as  many  as  559  ANP  units.16  However,  as  of  March  2010,  only  229  police 
units  were  being  directly  mentored  or  partnered  and  assessed  using  the  CM  system,  according  to  IJC 
reports.  IJC  and  NTM-A/CSTC-A  officials  agreed  that  the  charts  included  in  DOD's  June  2009  and 
October  2009  reports  to  Congress  on  Progress  toward  Security  and  Stability  in  Afghanistan  included  CM 
ratings  for  units  that  had  not  been  assessed.  Regarding  this  matter,  OSD  told  us  that  the  DOD  reports 
had  relied  on  data  from  CSTC-A  which  included  CM-4  ratings  for  units  that  had  been  provided 
equipment  or  training,  but  had  not  been  assessed  by  mentors  or  partners.  According  to  IJC  and  NTM- 
A/CSTC-A  officials,  a  continued  shortage  of  mentors  had  limited  police  development  efforts  to  the 
subset  of  police  districts  identified  as  most  important  to  achieve  campaign  plan  goals  and  in  which  FPDD 
or  In-District  Reform  (IDR)  training  had  been  provided  and  police  mentors/partners  were  present. 
Because  the  system  for  ANSF  assessment  was  under  review,  DOD  did  not  include  an  ANP  ratings  chart  in 
its  April  28,  2010,  report  to  Congress.17 

During  our  field  work,  we  found  that  a  number  of  police  units  were  being  mentored  but  not  assessed. 
Mentors  were  assigned  to  these  districts  but  no  assessment  reports  had  been  filed  because  the  ANP  unit 
had  not  received  FPDD  or  IDR  training.  IJC  was  unable  to  state  how  many  police  units  were  being 
mentored  but  not  assessed,  but  officials  at  Regional  Commands  South  and  West  identified  at  least  14 
police  districts  in  which  this  was  the  case. 

The  CM  assessment  system  relies  solely  on  the  reports  of  military  mentors/partners.  However,  civilian 
contractors  have  provided  training  and  mentoring  of  police  units  in  districts  where  military  personnel 
are  not  assigned.  According  to  an  IJC  official,  in  these  districts,  IJC  and  NTM-A/CSTC-A  did  not  track  the 
development  of  police  units.  For  instance,  contractors  have  provided  training  and  mentoring  for 
specialized  police  units,  including  Afghan  National  Civil  Order  Police  (ANCOP),  Afghan  Border  Police,  and 
Counter-Narcotics  Police,  none  of  which  had  been  routinely  assessed. 


16  The  ANP  includes  the  Afghan  Uniformed  Police  (AUP),  the  Afghan  National  Civil  Order  Police  (ANCOP),  the 
Afghan  Border  Police  (ABP),  as  well  as  specialized  units  such  as  counterterrorism  and  counternarcotics  police. 

17  DOD  is  required  to  report  to  Congress  every  180  days,  in  accordance  with  sections  1230  and  1231  of  the 
National  Defense  Authorization  Act  for  Fiscal  Year  2008  (Public  Law  110-181),  as  amended. 
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U.S.  AND  COALITION  ASSESSMENT  EFFORTS  HAVE  FACED  CHALLENGES 


A  variety  of  factors  affect  U.S.  and  Coalition  efforts  to  assess  ANSF  capabilities  and  the  consistency  and 
quality  of  assessments.  Key  challenges  have  included  poor  security  conditions,  shortages  of  police 
mentors,  and  inadequate  training  of  mentors  and  partners  in  assessment  methods. 

Security  Conditions  Have  Affected  Mentor  and  Partner  Assessment  Efforts 

According  to  IJC,  NTM-A/CSTC-A,  and  State/INL  officials  and  Dyncorp  civilian  police  advisers, 
Afghanistan's  non-permissive  security  environment  has  been  a  significant  challenge  for  ANP  mentoring 
and  assessment.  Increased  security  demands  in  many  areas  of  the  country  have  required  a  heavy  troop 
presence  to  provide  effective  mentoring  or  partnering  after  basic  police  training.  Mentoring  or 
partnering  has  been  more  risky  for  ANP  than  for  ANA  units,  given  IED  (improvised  explosive  device) 
threats  to  ground  movements  and  the  greater  dispersion  of  police  units,  which  has  required  mentor 
teams  to  "commute"  to  work.  Furthermore,  since  ANP  units  tend  to  be  relatively  small  compared  with 
ANA  units,  ANP  mentor/partner  teams  have  lacked  the  backup  protection  afforded  to  ANA 
mentors/partners  by  their  co-location  with  large  units  of  ANA  personnel. 

ANP  mentoring  and  assessment  has  been  significantly  limited  and  sometimes  ineffective  in  areas  where 
enemy  activity  is  intense.  In  some  cases,  force  protection  resources  were  insufficient  to  allow  for 
movement;  in  others,  mentoring  and  assessment  duties  were  challenged  by  the  conflicting  priorities  of 
commanders.  According  to  a  July  2009  report  by  civilian  police  advisers  in  Wazi  Zadran  police  district  of 
Paktia  Province,  the  mentor  team's  movements  were  restricted  due  to  diminished  manning  and 
challenging  security  conditions.  The  report  stated  that  mentors  had  not  been  able  to  visit  the  police  unit 
for  months,  observing,  "Mentoring  ultimately  turned  into  a  few  comments  or  statements  put  over  the 
radio  to  the  District  Center."  Narratives  within  U.S.  and  Coalition  police  assessment  reports  also  noted 
security  concerns,  citing  security-related  operational  constraints  as  a  challenge  to  mentoring  and 
assessment.  The  October  TRAT  participants  noted  that  mentors  had  experienced  difficulties  in  gaining 
access  to  police  posts,  with  some  locations  accessible  only  by  air. 

IJC  and  NTM-A/CSTC-A  officials  said  that  mentor/partner  performance  had  a  significant  effect  on 
development  and  assessment  practices.  Anecdotal  evidence  pointed  to  variation  in  mentor/partner 
efforts,  but  IJC  and  NTM-A/CSTC-A  did  not  attempt  to  assess  performance  systematically,  according  to 
these  officials.  TRAT  reports  and  officials  at  IJC  and  NTM-A/CSTC-A  raised  specific  concerns  about 
variations  in  the  quality  and  extent  of  the  mentoring  performed  by  NATO  mentor/partner  teams,  citing 
national  caveats  and  guidelines  as  constraining  factors  that  had  limited  the  efforts  of  some  NATO  teams. 

Mentor  and  Partner  Shortages  Have  Reduced  the  Effectiveness  of  Mentoring  and  Assessment 

The  ANSF  training  mission  has  been  challenged  by  longstanding  mentor/partner  shortfalls,  which  are 
likely  to  remain  an  issue  as  ANSF  growth  continues.  Shortfalls  have  been  particularly  severe  with  regard 
to  the  ANP  development  effort,  constraining  the  expansion  of  police  training  programs  as  well  as  the 
effectiveness  of  police  mentoring/partnering  after  basic  training.  As  of  March  29,  2010,  IJC  reported 
that,  while  all  U.S.  commitments  for  mentor/partner  teams  had  been  fulfilled,  NATO  had  failed  to 
deploy  the  38  army  mentor/partner  teams  and  163  police  mentor/partner  teams  required  to  meet  the 
demands  of  current  force  development  goals.  IJC  officials  told  us  they  could  not  provide  an  estimate  of 
these  shortfalls  in  terms  of  actual  numbers  of  personnel  required,  since  no  mandatory  personnel  size 


SIGAR  Audit-10-11  Security/ANSF  Capability  Ratings 


Page  15 


requirement  had  been  defined  for  NATO  mentor  and  partner  teams.  Nevertheless,  according  to  an  April 
2010  Department  of  Defense  report,  a  typical  NATO  mentor/partner  team  consists  of  40  personnel.18 
That  report  also  noted,  "Without  sufficient  mentors  and  trainers,  our  ability  to  effectively  grow  and 
develop  the  ANSF  is  at  risk." 

According  to  IJC  officials,  ANSF  assessments  had  been  submitted  each  month  without  any  evaluation  of 
the  level  of  access  mentors  and  partners  had  to  their  assigned  units.  Although  they  lacked  systematic 
measurements,  officials  at  IJC/ADAB  and  State/INL  said  they  knew  that  CM  ratings  for  some  ANP  units 
had  been  submitted  without  sufficient  mentor  or  partner  contact  to  support  the  assessment  results. 
Mentor  and  partner  teams  we  interviewed  agreed,  saying  they  were  typically  overstretched,  responsible 
for  mentoring/partnering  with  numerous  ANSF  units,  and  that  they  sometimes  had  to  neglect  certain 
ANSF  units  over  others.  An  October  2009  police  mentor  report  from  Moqur,  in  Ghazni  Province, 
acknowledged  that  low  mentor  coverage  at  that  district  made  the  rating  suspect  and  their  ability  to 
advise  and  assist  problematic.  In  another  example,  one  Kabul  police  mentor  team  that  received  an 
additional  assignment  to  train  a  problematic  ANA  kandak,  resorted  to  occasionally  calling  its  police 
districts  rather  than  observing  them  directly.  The  team  had  continued  to  submit  CM  assessments  under 
these  conditions  but  said  that  its  ability  to  mentor  and  assess  effectively  had  been  seriously  impaired. 

Mentors  and  Partners  Lack  Adequate  Training 

Military  personnel  serving  as  ANSF  mentors  and  partners  lack  adequate  training  to  support  their 
mission,  according  to  mentor  and  partner  teams  and  IJC/VTT  officials.  ANP  mentors  stated  that  they 
found  that  they  were  unprepared  for  their  duties  and  unfamiliar  with  MOI  regulations  concerning  ANP 
operations.  Mentors/partners  reported  that  insufficient  guidance  and  training  made  it  difficult  to  assess 
police  districts'  compliance  with  MOI  decrees  and  guidance,  as  required  in  numerous  questions  on  the 
CE  checklist.  In  response  to  our  written  questions,  mentors  in  RC-North  reported,  "The  fundamental  lack 
of  training  of  [police  mentor  teams]  in  essential  MOI  decrees,  procedures,  and  common  problems 
hampers  the  [police  mentor  team's]  ability  to  provide  meaningful  information  and  the  lack  of  a 
comparable  standard  creates  a  large  degree  of  subjectivity  from  one  team  to  the  next."  According  to 
IJC/VTT,  army  mentors  have  faced  similar  challenges,  including  the  lack  of  training  on  ANA  systems  and 
Ministry  of  Defense  (MOD)  decrees  and  guidance. 

To  support  ANSF  assessors'  use  of  the  CE  checklist,  NTM-A/CSTC-A  produced  a  video  training  program 
and  an  instruction  manual.  However,  the  mentors/partners  we  interviewed  criticized  the  video  and 
other  materials,  stating  that  they  were  focused  on  technical  aspects  of  completing  the  form,  rather  than 
on  assessment  practices.  The  assessors  said  that  to  improve  the  quality  and  consistency  of  their 
evaluations  they  needed  additional  guidelines  to  enhance  their  understanding  of  assessment  criteria 
and  best  practices. 


18  DOD,  Progress  toward  Security  and  Stability  in  Afghanistan,  April  2010,  p.  17. 
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SYSTEMIC  AIMSF  DEFICIENCIES  HAVE  UNDERMINED  EFFORTS  TO  DEVELOP  UNIT  CAPABILITIES 


Systemic  ANSF  deficiencies  have  undermined  progress  toward  developing  independent  Afghan  security 
forces.  These  deficiencies  include  logistics  problems,  personnel  attrition,  inadequate  personnel 
authorizations,  infrastructure  challenges,  corruption,  and  drug  abuse  and  illiteracy.  In  commenting  on  a 
draft  of  this  report,  NTM-A/CSTC-A  stated  it  is  aware  of  these  deficiencies  and  is  working  to  address 
them  (see  appendix  III). 

Severe  Logistics  Problems  Challenge  the  Development  of  Self-Sustaining  Operations 

Deficiencies  in  ANSF  logistics  have  challenged  U.S.  and  Coalition  efforts  to  develop  independent  army 
and  police  forces  for  Afghanistan.  The  CM  assessment  system  has  included  measures  designed  to 
evaluate  an  army  or  police  unit's  ability  to  carry  out  logistics  functions  crucial  to  self-sustainment. 
However,  officials  of  IJC,  NTM-A/CSTC-A,  State/INL,  ANSF  partners  and  mentors,  VTT  teams,  employees 
of  DynCorp,  and  Afghan  army  and  police  commanders  uniformly  described  above-unit-level  problems 
with  logistics  support  functions  as  significant  impediments  to  the  development  of  self-sufficient  ANSF 
units.  At  every  ANSF  unit  we  visited,  army  and  police  alike— including  top-rated  CM1  units— we  found 
logistics  supply  problems  directly  interfering  with  unit  operations.  Army  and  police  mentors  and 
partners  for  all  18  units  we  visited  reported  that  logistics  support  was  a  severe  problem  and  supply- 
request  systems  non-functional.  Assessment  reports  we  reviewed  for  other  units  echoed  this  concern. 
Mentors  and  partners  reported  vanishing  supply  requests,  hoarding  at  provincial  and  Corps  levels,  and 
an  inability  to  track  supply-request  forms  through  the  ministerial  supply  chain  systems. 

Police  mentors  and  partners  told  us  that  logistics  failures  were  not  only  affecting  operational 
effectiveness  and  life  support,  but  also  challenging  efforts  to  develop  ANSF  confidence  in  ministerial 
leadership  and  draining  ANSF  morale.  Police  mentors  in  Kunduz  Province  wrote  in  a  February  2010 
statement,  “The  logistics  system  does  not  function  to  any  acceptable  level...  [We]  would  hesitate  to 
state  that  a  logistics  system  even  exists."  In  another  example,  an  ANP  mentor  in  RC-North  noted,  "I 
would  think  [the  ANP]  would  be  much  more  eager  to  be  independent  of  coalition  support  if  they 
thought  their  own  system  was  functional  at  all.  If  their  own  chain  of  command  is  broken,  what 
motivation  is  there  to  have  us  leave?  Right  now  we  are  the  source  of  all  repairs  and  most  other  logistics 
support." 

ANA  mentors  and  partners  also  reported  severe  deficiencies  associated  with  the  MOD  supply  system. 
NATO  mentors  in  Kunduz  Province  described  the  system  as  a  "fill  or  kill"  process  in  which  requests  are 
submitted,  and  then  disappear.  In  other  cases,  they  said  supplies  were  pushed  down  from  MOD,  and 
thus  were  not  responsive  to  the  units'  needs.  A  November  2009  assessment  report  on  a  CMl-rated 
army  unit,  the  1st  kandak  2nd  Brigade  of  the  203rd  Corps,  reported  that  the  supply  process  was  a  failure. 
The  kandak  received  no  feedback  on  the  status  of  its  supply  requests  and  was  primarily  dependent  on 
U.S.  soldiers  to  remedy  supply  issues. 

NTM-A/CSTC-A  agreed  that  logistics  challenges  are  of  great  concern  and  stated  it  has  taken  a  variety  of 
steps  to  address  known  shortcomings,  including  making  plans  to  field  and  develop  additional  ANSF 
logistics  enablers  beginning  in  FY  2011. 
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Heavy  Personnel  Attrition  Diminishes  Training  Gains  and  Operational  Effectiveness 

Heavy  attrition  in  the  Afghan  military  and  police  forces  has  undermined  developmental  gains  in  many 
units,  by  reducing  the  number  of  trained  personnel  available  for  operations.  As  of  March  2010,  the 
ANA's  AWOL  rate  was  12  percent  and,  as  of  May  2010,  the  ANP  had  an  overall  attrition  rate  of  over  17 
percent.  Those  losses,  coupled  with  high  levels  of  approved  absence  and  frequent  reassignments,  have 
left  many  units  with  insufficient  personnel  to  permit  effective  operations.  For  example,  according  to 
NTM-A/CSTC-A,  as  of  February  2010,  solar-year-to-date  attrition  in  ANCOP— "the  premier  force  in  the 
ANP,"  according  to  DOD— was  about  73  percent.  One  ANCOP  battalion  in  RC-West  had  an  attrition  rate 
of  140  percent  over  that  period.19  The  attrition  rate  for  Afghan  Border  Police  over  the  same  period  was 
27.4  percent,  according  to  NTM-A/CSTC-A. 

In  many  Afghan  Uniformed  Police  units,  attrition  has  drained  a  large  proportion  of  those  originally 
trained  in  the  FPDD  program.  We  found  numerous  examples  of  this  within  ANP  assessment  reports.  In 
one  instance,  in  July  2009,  mentors  reported  that  at  Waza  Zadran  police  district  in  Paktia  Province  there 
were  only  6  ANP  personnel  in  place,  compared  with  the  tashkil  authorization  of  53  officers.  That  district 
completed  FPDD  training  in  April  2008,  when  23  students  were  trained,  according  to  NTM-A/CSTC-A's 
records.  Mentors  said  this  severe  attrition  was  largely  due  to  actions  taken  by  powerful  anti-coalition 
forces  and  disappointment  over  pay  levels.  NTM-A/CSTC-A  documents  we  reviewed  showed  that  Waza 
Zadran  had  attained  a  top-level  capability  rating  and  had  even  been  slated  for  graduation  from  the  FPDD 
program,  as  late  as  April  2009.  However,  by  February  2010,  the  district  had  dropped  to  the  lowest 
rating  level,  CM4. 

Inadequate  Personnel  Authorizations  Have  Inhibited  Effectiveness 

The  goals  against  which  ANSF  manning  levels  have  been  assessed  may  not  represent  end  states 
adequate  to  permit  independence.  Under  the  CM  rating  system,  an  ANSF  unit  could  earn  a  top  CM1 
rating  for  personnel,  if  it  had  a  full  complement  of  personnel  assigned,  as  compared  with  the  ministerial 
tashkil  authorization.  However,  according  to  mentors/partners  and  Afghan  officials  we  interviewed,  and 
assessment  reports,  some  ministerial  personnel  authorizations  have  been  inadequate  to  support 
effective  operations.20  For  both  army  and  police  forces,  the  ability  of  units  to  support  mobile  operations, 
including  patrols  and  quick  reaction  force  operations  has  been  limited  by  the  number  of  personnel  lost 
through  attrition  and  reassignment  and  the  number  needed  to  man  checkpoints  and  outposts. 

Infrastructure  Challenges  Affect  ANSF  Morale,  Security,  and  Operational  Effectiveness 

We  identified  widespread  deficiencies  in  ANSF  infrastructure,  including  problems  directly  affecting  army 
and  police  operations.  Facilities  were  often  unable  to  support  the  basic  needs  of  assigned  personnel, 
lacking  running  water,  electricity,  and  proper  billeting  and  dining  facilities.  In  many  cases,  police  chiefs 
were  choosing  to  use  fuel  to  run  generators,  rather  than  conduct  patrols.  Facilities  problems  were 
directly  affecting  the  morale  and  operational  readiness  of  fielded  units.  We  observed  serious 
infrastructure  issues  at  many  units  we  visited,  including  those  rated  CM1.  In  addition  to  basic  life 


19  DOD  reported  in  April  2010  that  NTM-A/CSTC-A,  Mol,  and  EUPOL  have  established  an  ANCOP  Working  Group  to 
examine  how  to  reduce  attrition  and  improve  the  ANCOP  program. 

20  Similar  observations  regarding  insufficient  police  authorizations  were  recorded  in  the  September  2009  DOD 
Inspector  General  report,  SPO-2009-007,  Report  on  the  Assessment  of  U.S.  and  Coalition  Plans  to  Train,  Equip,  and 
Field  the  Afghan  National  Security  Forces,  p.  112. 
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support  issues,  infrastructure  problems  have  raised  security  concerns.  At  police  units  we  visited, 
including  those  rated  CM1,  we  observed  examples  of  inadequate  security  perimeters  and  unsecured 
weapons  storage  areas.  Mentor  and  partner  reports  frequently  noted  that  facilities  issues  were  holding 
back  development  and  raised  concern  about  units'  ability  to  operate  independently,  without  long-term 
solutions  to  facilities  maintenance  issues.  An  October  2009  assessment  report  from  Dand  Patan  police 
district  in  Paktia  Province  noted,  "Facility  maintenance  is  critical...  MOI  must  create  a  long  term  solution 
for  this  issue  however  if  ANP  are  ever  to  become  truly  autonomous." 

Mentors  and  partners  acknowledged  that  widespread  efforts  have  been  under  way  by  the  United  States 
and  Coalition  partners  to  help  improve  ANSF  facilities.  Nevertheless,  they  also  reported  pervasive 
delays  in  construction  projects.  Construction  delays  of  over  a  year  at  the  2/209th  brigade  garrison 
project  we  visited  in  Kunduz  Province  had  left  four  kandaks  housed  in  temporary  facilities,  dramatically 
restricting  training  and  operations. 21  During  our  February  2010  visit,  we  observed  Afghan  army 
personnel  coping  with  deep  mud,  freezing  conditions,  unsanitary  shower  and  bathroom  facilities, 
inadequate  dining  facilities,  medically  unsafe  food  storage,  and  sewage  being  openly  discharged  on  the 
surface  of  the  compound. 

Corruption  Has  Undermined  the  Development  of  ANSF  Independence 

In  addition  to  its  corroding  effect  on  public  confidence  in  ANSF  and  thereby  the  Afghan  government, 
corruption  was  cited  by  army  and  police  mentors  and  partners  as  a  key  factor  undermining 
developmental  progress  and  morale  at  the  unit  level.  ANP  mentors  and  State/INL  officials  we 
interviewed  expressed  doubts  about  the  validity  of  top  ratings  in  certain  police  districts,  due  to  the 
effects  of  corruption  problems.  Mentors  and  partners  we  interviewed  and  assessment  reports  we 
reviewed  observed  that  ANSF  corruption  has  affected  the  leadership  of  many  units,  disrupted  supply 
chain  operations,  and  diminished  willingness  to  cooperate  with  mentoring/partnering  efforts  in  some 
cases.  For  example,  in  a  February  2010  TRAT  report  on  ANP  capabilities,  RC-East  commanders 
commented,  "Corruption  of  Senior  ANP  Leaders  continues  to  thwart  all  of  our  developmental  efforts." 
RC-West  commanders  also  weighed  in,  reporting,  "Corruption  is  continuing  to  plague  the  Western  ANP, 
and  this  includes  an  increase  of  influence  by  the  [insurgents']  shadow  government." 

Corruption  has  had  direct  negative  effects  on  daily  ANSF  operations  in  the  following  areas: 

•  Supply  chain:  Police  assessments  prepared  in  October  2009  and  January  2010  cited  multiple 
instances  of  Afghan  police  officials  selling  ANP  materials,  taking  money  from  transport  trucks  at 
checkpoints,  skimming  fuel,  and  pilfering  supplies,  including  weapons. 

•  Fuel:  Corruption  has  affected  fuel  availability  and  distribution  within  the  ANSF,  and  thus  the 
operational  effectiveness  of  army  and  police  units.  Problems  with  fuel  accountability,  including 
siphoning  and  selling  fuel,  were  widely  reported  in  police  unit  assessments  and  detailed  for  us 
on  our  visits  to  army  and  police  units  around  the  country.  An  October  2009  report  on  a  top- 
rated  police  unit  in  Chaparhar,  Nangarhar  Province,  stated  that  the  lack  of  accountability  for  fuel 


21SIGAR  conducted  a  separate  audit  of  contract  performance  and  oversight  for  the  construction  of  the  ANA 
garrison  at  Kunduz.  See  SIGAR,  ANA  Garrison  at  Kunduz  Does  Not  Meet  All  Quality  and  Oversight  Requirements; 
Serious  Soil  Issues  Need  to  Be  Addressed,  10-09,  April  20,  2010. 
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was  the  greatest  issue  in  sending  out  patrols.  A  January  2010  report  from  Bala  Boluk,  Farah 
Province,  described  fuel-related  corruption  as  a  "perpetual  problem." 

•  Illegal  checkpoints:  Mentor/partner  reports  noted  that  in  some  places  it  was  typical  for  Afghan 
police  to  "shake  down"  travelers  for  money  in  order  to  pass  through  districts. 

•  Personnel:  Misuse  and  diversion  of  police  personnel  were  cited  in  numerous  police  assessments 
as  challenges  to  effective  operations,  weakening  the  present-for-duty  strength  of  many  police 
units. 

•  Pay  and  rank:  Pay  fraud  and  rank  abuse  were  widely  reported,  although  electronic  fund  transfer 
programs  were  credited  with  reducing  payroll  abuse  in  some  police  units.  Reports  of  ghost 
police  were  common  and  RC-North  police  mentors  stated  that  paying  for  promotion  was 
commonly  accepted  among  the  ANP.  Mentors  in  Ghowrmach  police  district,  Baghdis  Province, 
reported  in  October  2009  that,  even  though  electronic  fund  transfer  was  not  available  for  the 
unit  due  to  its  remote  location,  pay  continued  for  absent  police,  those  whose  loyalties  were 
suspect,  and  officers  who  had  never  worked  in  the  district. 

Corruption  also  has  directly  affected  the  ability  of  U.S.  and  Coalition  forces  to  mentor  and  partner 
effectively  with  ANSF  units.  Mentors  and  partners  we  interviewed  said  corruption  at  the  police  chief  or 
kandak  commander  level  had  undermined  their  efforts  to  develop  ANSF  units.  NTM-A/CSTC-A  stated 
that  it  has  taken  actions  to  combat  corruption,  including  steps  such  as  assisting  MOI  in  conducting  a 
Personnel  Asset  Inventory  (PAI)  and  implementing  Electronic  Funds  Transfer  for  ANSF  pay  delivery. 

Drug  Abuse  and  Illiteracy  Continue  to  Challenge  Operational  Effectiveness 

Drug  abuse  continues  to  be  a  problem  for  the  Afghan  army  and  police.  As  of  February  2010,  results  from 
drug  tests  during  a  personnel  asset  inventory  of  the  Afghan  police  force  overall  showed  positive  results 
for  drug  use  in  17  percent  of  police  tested.  NTM-A/CSTC-A  officials  who  assisted  MOI  with  the  inventory 
said  that  this  rate  was  likely  understated,  and  the  March  TRAT  report  noted  that  the  inventory  results 
for  Ghazni  and  Paktika  Provinces  were  returning  an  average  usage  rate  for  controlled  substances  of  at 
least  50  percent,  with  the  highest  rates  in  more  remote  areas.  In  addition,  assessment  reports  and 
mentors  and  partners  at  many  of  the  army  and  police  units  we  visited  cited  instances  of  drug  and 
alcohol  abuse.  According  to  several  officials  with  responsibility  for  ANSF  development,  an  extreme  case 
of  drug  abuse  had  occurred  at  an  ANCOP  unit  of  about  100  personnel  based  at  Nimla  Gardens, 

Nangarhar  Province.  There,  according  to  eye  witness  accounts  from  U.S.  military  personnel,  ANCOP 
personnel  were  openly  using  marijuana  and  were  unwilling  to  conduct  operations  or  even  leave  their 
compound. 

Although  steps  have  been  taken  to  introduce  literacy  training  for  the  Afghan  army  and  police,  high  levels 
of  illiteracy  in  both  forces  continue  to  hamper  development.  Army  and  police  mentors/partners  stated, 
and  assessment  reports  corroborated,  that  illiteracy  has  constrained  the  development  of  support 
systems  such  as  personnel  management  and  logistics,  including  accountability  for  equipment  and  fuel. 
Widespread  illiteracy  was  also  cited  as  an  impediment  to  the  development  of  intelligence  and  navigation 
capabilities.  In  response  to  the  problem  of  ANSF  illiteracy,  NTM-A/CSTC-A  has  made  literacy  training 
mandatory  for  all  ANSF  soldiers  and  police  and  continues  to  increase  the  number  of  literacy  instructors. 
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CONCLUSIONS 


Weaknesses  in  design  and  implementation  of  the  CM  rating  system  have  led  to  variable  results  and  have 
impeded  a  clear  understanding  of  the  operational  effectiveness  of  ANSF  units.  The  Department  of 
Defense  and  ISAF  have  expressed  concerns  about  the  risk  of  using  an  assessment  system  that  provides 
an  ineffective  measure  of  performance.  Acknowledging  the  risk  such  a  system  poses,  DOD  reported  in 
late  April  2010  that  General  McChrystal  was  considering  alternatives  to  CM  ratings.  According  to  IJC 
officials,  the  CM  rating  is  being  replaced  with  a  new  unit-level  assessment  system.  Moving  forward,  it  is 
critical  that  whatever  assessment  system  is  used  to  evaluate  ANSF  units  should  provide  consistent  and 
reliable  reporting  of  operational  effectiveness.  Without  such  measures,  decision  makers  will  not  have  a 
clear  understanding  of  the  extent  to  which  progress  is  being  made  in  developing  Afghan  security  forces 
capable  of  independently  conducting  operations,  and  ultimately  securing  Afghanistan. 


RECOMMENDATIONS 

We  are  including  ten  recommendations  for  ISAF  Joint  Command  (IJC)  and  three  for  NATO  Training 
Mission-Afghanistan/Combined  Security  Transition  Command-Afghanistan  (NTM-A/CSTC-A)  intended  to 
improve  the  reliability  of  future  ANSF  assessments  and  to  support  U.S.  and  international  efforts  to 
develop  ANSF  capabilities. 

Recommendations  for  IJC: 

To  improve  measurement  and  validation  techniques  in  order  to  obtain  more  reliable  assessments  of 
ANSF  operational  effectiveness,  we  recommend  that  IJC: 

1.  Place  greater  emphasis  on  subjective  assessment  of  overall  operational  capability  provided 
by  mentor  and  partner  team  evaluators,  including  a  requirement  to  complete  narrative 
observations. 

2.  Support  the  completion  of  meaningful  and  consistent  evaluations,  by  developing  a  manual 
for  mentor  and  partner  team  assessors  that  incorporates  real-world  examples  and  best 
practices. 

3.  Work  with  NTM-A/CTSC-A  to  provide  additional  training  and  guidance  to  mentor  and 
partner  teams  regarding  the  nature  of  MOI  and  MOD  decrees  and  procedures. 

4.  Encourage  an  equivalent  understanding  among  raters  of  different  ANSF  units  by  ensuring 
rating-level  definitions  are  consistent. 

5.  Incorporate  present-for-duty  figures  as  a  percentage  of  assigned  personnel  into  assessments 
for  both  police  and  army  units.  Note:  Until  reliable  present-for-duty  numbers  are  available 
for  police  units,  we  recommend  that  IJC  incorporate  a  separate  measurement  for  tracking 
"available"  personnel  who  have  been  detailed  to  other  locations  (i.e.,  available,  but  not 
present). 

6.  Conduct  independent  validations  of  top-level  Afghan  police  unit  capabilities. 


SIGAR  Audit-10-11  Security/ANSF  Capability  Ratings 


Page  21 


To  improve  the  transparency  and  accuracy  of  ANSF  assessment  reporting,  we  recommend  that  IJC: 

7.  Begin  to  systematically  track  the  number  of  unit-level  evaluations  not  submitted  each 
month  and  the  number  of  elements  not  observed  within  those  assessments. 

8.  Add  an  additional  overall  rating  level  to  represent  units  "not  assessed"  where  no 
mentor/partners  are  assigned. 

To  maximize  visibility  into  Afghan  police  capabilities,  we  recommend  that  IJC: 

9.  Require  all  U.S.  and  Coalition  military  mentor  and  partner  teams  to  submit  unit  assessments 
and  that  IJC  begin  requesting  unit  assessments  from  civilian  advisers  in  those  districts  where 
military  mentors/partners  are  not  assigned. 

To  counteract  perverse  incentives  resulting  from  the  application  of  the  ANSF  assessment  system,  we 
recommend  that  IJC 

10.  Evaluate,  in  conjunction  with  NTM-A/CSTC-A,  MOI,  and  MOD,  the  creation  of  an  incentive 
system  to  reward  Afghan  units  for  the  development  of  validated,  top-level  operational 
capabilities. 


Recommendations  for  NTM-A/CSTC-A: 

To  help  develop  compliance  with  ministerial  logistics  systems  and  increase  the  transparency  of  ANSF 
logistics  operations,  we  recommend  that  NTM-A/CSTC-A: 

1.  Work  with  MOI  and  MOD  to  implement  functional  tracking  systems  for  monitoring  supply 
requests. 

To  counterbalance  the  effects  of  attrition  in  FPDD  units,  we  recommend  that  NTM-A/CSTC-A: 

2.  Develop  a  program  to  systematically  provide  backfill  training  for  new  recruits  assigned  to 
previously-trained  FPDD  districts. 

To  facilitate  the  training  of  as  many  drivers  as  possible  and  improve  operational  effectiveness  at  the  unit 
level,  we  recommend  that  NTM-A/CSTC-A: 

3.  Implement  a  train-the-trainer  driving  program  to  allow  each  ANSF  unit  (district  or  kandak) 
to  extend  driver  training  using  its  own  personnel. 
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COMMENTS 


IJC,  NTM-A/CSTC-A,  OSD  Policy,  and  OSD  Personnel  and  Readiness  provided  written  comments  on  a 
draft  of  this  report.  These  comments  are  provided  in  appendix  II,  III,  IV,  and  V,  respectively,  along  with 
our  response  to  the  comments. 

In  its  response,  IJC  concurred  or  partially  concurred  with  ten  recommendations.  As  noted  in  our  report, 
in  April  2010,  IJC  adopted  a  new  ANSF  assessment  system.  IJC  stated  in  its  response  to  our  report  that  it 
intended  to  address  our  recommendations  as  part  of  its  implementation  of  that  new  system,  the 
Commander's  Unit  Assessment  Tool  (CUAT).  IJC  disagreed  with  two  recommendations,  which  we  have 
removed  from  the  final  report  because  IJC's  responses  fulfilled  the  spirit  of  our  recommendations.  In 
commenting  on  a  draft  of  our  report,  IJC  said  our  report  should  have  great  utility  and  described  our 
work  as  accurate,  well-informed,  and  comprehensive. 

In  its  response,  NTM-A/CSTC-A  concurred  with  our  recommendations,  but  suggested  that  our  report 
included  information  that  was  out  of  date.  Our  audit  work  was  conducted  from  October  2009  through 
May  2010  and  relied  on  interviews  and  information  collected  over  that  entire  period,  including  field 
observations  of  18  ANSF  units  made  during  February  and  March  2010  and  detailed  reviews  of  ANSF 
assessment  documents  dated  through  March  29,  2010.  NTM-A/CSTC-A  provided  additional  information 
regarding  progress  in  ANSF  development,  including  its  180  Day  Internal  Review  and  Way  Forward,  dated 
May  29,  2010.  Lieutenant  General  Caldwell,  Commander  of  NTM-A  and  CSTC-A,  stated  that  the 
additional  materials  contain  important  information  regarding  NTM-A/CSTC-A's  efforts  and  challenges 
ahead. 

In  its  response,  OSD  Policy  provided  additional  information,  including  clarification  about  the  nature  of 
DOD  reports  to  Congress  and  comments  regarding  DOD's  concerns  about  the  CM  system.  We  included 
some  of  this  information  in  our  report. 

OSD  Personnel  and  Readiness  also  commented  on  our  report,  and  provided  additional  information 
regarding  the  history  of  efforts  to  improve  ANSF  assessments,  including  details  on  OSD's  involvement  in 
attempting  to  address  reliability  and  integrity  problems  with  the  CM  rating  system.  We  have  included 
the  additional  information  provided  in  appendix  V  of  our  report. 
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APPENDIX  I:  SCOPE  AND  METHODOLOGY 


This  report  provides  the  results  of  the  Office  of  the  Special  Inspector  General  for  Afghanistan 
Reconstruction's  review  of  the  assessment  system  used  to  rate  ANSF  capabilities. 

To  evaluate  the  reliability  of  the  CM  rating  system,  we  reviewed  documentation  from  and  conducted 
interviews  with  responsible  officials  from  IJC,  NTM-A/CSTC-A,  State/INL,  OSD/Policy,  Embassy  Kabul, 
CENTCOM,  DynCorp,  MPRI,  Border  Management  Taskforce,  Provincial  Reconstruction  Team  Kunduz, 

Task  Force  Mountain  Warrior,  RC-East  ANSF  Development  Section,  Afghan  army  and  police 
commanders.  Additionally,  to  obtain  firsthand  evidence  regarding  the  capabilities  of  fielded  Afghan 
army  and  police  units  and  the  evaluation  of  those  capabilities  at  the  kandak  and  district/precinct  level, 
we  performed  direct  observations  at  18  Afghan  army  and  police  units  in  the  following  locations:  Afghan 
Uniformed  Police  Kabul  Districts  1,  3,  7  and  11;  Aliabad  Police  District,  Kunduz  Province;  Kunduz  City 
Precincts  2,  3,  and  4  plus  associated  checkpoints  and  outposts;  Kameh  Police  District,  Nangarhar 
Province;  Bati  Kot  Police  District,  Nangarhar  Province;  Provincial  Police  Headquarters  Nangarhar 
Province;  ANCOP  3rd  Kandak  Headquarters,  Nangarhar  Province;  OCC-P  Nangarhar  Province;  ANA  2nd 
BDE,  209th  Corps,  including  3  infantry  kandaks  and  one  CSS  kandak;  and  ANA  3rd  BDE,  201st.  At  all 
locations,  to  understand  challenges  facing  assessment  efforts  and  concerns  regarding  the  assessment 
system  we  interviewed  mentors  and  partners— including  U.S.  and  Coalition  forces— assigned  to  work 
with  the  Afghan  units.  Our  interviews  with  NATO  mentors  and  partners  included  military  personnel  from 
Germany,  Belgium,  and  Canada.  At  every  army  and  police  unit  we  visited,  we  also  interviewed  Afghan 
officials.  To  understand  the  role  of  civilian  police  advisers  in  ANP  mentoring/partnering  and  evaluation, 
we  met  with  officials  of  State/INL  in  Washington,  D.C.  and  Kabul,  as  well  as  officials  of  DynCorp  in 
Afghanistan.  We  also  met  with  MPRI,  to  obtain  information  about  that  company's  role  in  supporting 
ANSF  development,  including  systems. 

To  examine  ANSF  assessment  procedures,  including  methods  of  analysis  and  reporting,  we  held  several 
meetings  each  with  officials  from  NTM-A/CSTC-A  CJ5  and  IJC  Assessments.  We  also  met  on  several 
occasions  with  IJC's  ANSF  Development  Assistance  Bureau  to  understand  the  system  by  which 
assessment  reporting  has  been  processed  at  IJC.  We  also  met  with  the  ANSF  Development  Section  for 
RC-East,  and  the  ANSF  Coordinator  for  Taskforce  Mountain  Warrior  in  Nangarhar  Province.  To  examine 
ANSF  development  efforts  as  they  related  to  assessment  practices,  in  Kabul,  we  met  with  NTM-A/CSTC- 
A's  Command  Training  Advisory  Group— Army  and  Command  Training  Advisory  Group— Police,  NTM- 
A/CSTC-A  CJ7  (Force  Integration  and  Training),  NTM-A/CSTC-A  CJ5,  NTM-A/CSTC-A  CJ1,  and  the  Border 
Management  Task  Force.  We  met  with  NTM-A/CSTC-A's  Office  of  Security  Cooperation  regarding  efforts 
to  improve  weapons  accountability.  We  also  attended  TRAT  meetings  at  Camp  Phoenix  and  IJC  HQ,  in 
October  2009  and  February  2010  and  conducted  2  meetings  with  members  of  IJC's  Validation  Transition 
Team,  in  October  2009  and  February  2010. 

We  performed  documentary  reviews  of  TRAT  reports  for  both  the  Afghan  army  and  police  for  each 
month  from  October  2009  through  March  2010.  We  also  reviewed,  for  qualitative  and  quantitative 
information,  87  CE  Checklist  file  submissions  for  Afghan  police  districts  or  precincts  as  submitted  in 
October  2009,  and  113  such  files  as  submitted  in  January  2010.  We  were  unable  to  obtain  a  comparable 
set  of  Afghan  army  assessment  submission  files,  but  reviewed  an  incomplete  sample  of  assessment 
submissions  representing  examples  from  October  2009,  November  2009,  and  January  2010.  Our  review 
also  included  96  narrative  district  reports  regarding  Afghan  Uniformed  Police  units,  prepared  by 
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DynCorp  civilian  advisors.  To  determine  the  extent  of  regression  amongst  ANSF  units,  we  reviewed  all 
available  CM  ratings  for  ANA  and  ANP  units  from  February  2009  through  January  2010.  We  counted  the 
occurrence  of  rating-level  regressions  during  this  period,  looking  at  declines  from  any  level  and  also 
counting  declines  from  the  CM1  rating  level.  Since  our  analysis  required  at  least  two  data  points  for 
comparison,  we  included  only  those  ANA  and  ANP  units  that  had  at  least  two  CM  ratings  during  this 
period. 

To  examine  the  effects  of  challenges  on  U.S.  and  Coalition  assessment  efforts  and  to  examine  systemic  , 
ANSF  deficiencies  that  have  undermined  efforts  to  develop  unit-level  capabilities,  we  reviewed 
documentation  from  and  conducted  interviews  with  responsible  officials  from  IJC,  NTM-A/CSTC-A, 
State/INL,  DynCorp,  Task  Force  Mountain  Warrior,  RC-East  ANSF  Development  Section,  U.S.  and 
Coalition  army  and  police  mentors  and  partners,  and  Afghan  army  and  police  commanders.  We  also 
made  observations  while  visiting  the  ANSF  units  listed  above.  We  also  performed  detailed  reviews  of 
assessment  records  for  both  Afghan  army  and  police  units,  as  detailed  above. 

We  conducted  work  in  Afghanistan  and  Washington,  D.C.,  from  October  2009  through  May  2010  in 
accordance  with  generally  accepted  government  auditing  standards.  These  standards  require  that  we 
plan  and  perform  the  audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe  the  evidence  obtained  provides 
a  reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit  objectives.  The  audit  was 
conducted  by  the  Office  of  the  Special  Inspector  General  for  Afghanistan  Reconstruction  under  the 
authority  of  Public  Law  No.  110-181,  and  the  Inspector  General  Act  of  1978,  as  amended. 
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APPENDIX  II:  COMMENTS  FROM  ISAF  JOINT  COMMAND 


Headquarters 

International  Security  Assistance  Force  Joint  Command 
Kabul,  Afghanistan 
APO  AE  09354 


IN  REPLY  REFER  TO 
IJC  CMD  GRP 


15  June  2010 

MEMORANDUM  FOR  SPECIAL  INSPECTOR  GENERAL  FOR  AFGHANISTAN 
RECONSTRUCTION 


SUBJECT :  ISAF  Joint  Command  Comments  on  SIGAR  Draft  Report  on  Afghan  National 
Security  Forces  Assessments 

Enclosed  are  the  ISAF  Joint  Command’s  comments  on  the  subject  report.  We  appreciate 
the  effort  your  team  put  in  to  produce  their  well-informed  and  comprehensive  audit  of  this 
critical  process.  Thank  you  for  providing  this  opportunity  to  review  and  comment  on  their  work. 

As  a  point  of  clarification  concerning  the  NTM-A  response  to  SIGAR,  IJC  is  not 
considering  phasing  out  Focused  Police  District  Delivery  (FPDD)  training.  The  FPDD  program 
is  critical  to  training  the  thousands  of  police  officers  who  are  in  the  field  without  formal  training. 


A 


DAVID  M.  RODRIGUEZ 


Lieutenant  General,  U.S.  Army 
Commander, 

International  Security  Assistance  Force 
Joint  Command 


Enclosure: 

1  —  IJC  Comments 
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SIGAR  DRAFT  REPORT  -  DATED  June  2010 
SIGAR  Audit  10-11 

"Actions  Needed  to  Improve  the  Reliability  of  Afghan  Security  Forces 

Assessments" 

DC  COMMENTS 
TO  THE  DRAFT  REPORT 


To  improve  measurentent  and  validation  techniques  in  order  to  obtain  more 
reliable  assessments  of  ANSF  operational  effectiveness,  we  recommend  that  DC: 


RECOMMENDATION  L  (page  19.  SIGAR  Draft) 

Pace  greater  errphasis  on  subjective  assessment  of  overall  operational  capability  provided 
by  mentor  and  partner  team  evaluators,  including  a  requirement  to  complete  narrative 
observations. 

DC  RESPONSE:  DC  concurs  with  and  has  implemented  this  recomniendation.  The 
new  Commander's  Unit  Assessment  Tool  (CUAT)  was  specifically  designed  to  allow 
the  partners  and  the  commanders  to  provide  subjective  assessments  in  a  narrative 
format. 


RECOMMENDATION  2.  (page  19.  SIGAR  Draft) 

Support  the  completion  of  meaningful  and  consistent  eva  u  at  ions,  by  developing  a  manual 
for  mentor  and  partner  team  assessors  that  incorporates  real-world  examples  and  best 
practices. 


DC  RESPONSE:  DC  concurs  with  and  will  implement  this  recomniendadon.  IJC- 
ADAB  has  prepared  a  manual  on  how  partners,  mentor  and  commanders  will  use 
the  CUAT  and  we  will  incorporate  real-world  examples  and  best  practices  in  the 
revised  version  we  are  currently  preparing. 


RECOMMENDATION  3.  (page  19,  SIGAR  Draft! 

Work  with  NTM-A'CTSC-A  to  provide  additional  training  and  guidance  to  mentor  and  partner 
teams  regarding  the  nature  of  MOl  and  MOO  decrees  and  procedures. 


I3C  RESPONSE:  DC  concurs  with  and  will  implement  this  recommendation.  LJC 
will  ensure  that  information  concerning  the  nature,  importance  and  location  of 
MOI  and  MOD  decrees  is  part  of  our  RIP/TOA  checklist  and  that  OMLT / POMLT 
teams  are  provided  this  information  when  they  receive  initial  training  in  theater. 


RFCOMMFNDATION  4.  (page  10.  SIGAR  Draftl 

Encourage  an  eqjiva  ent  understand  ng  among  raters  of  different  ANSF  units  by  ensuring 
rating-level  definitions  are  consistent. 

DC  RESPONSE:  DC  concurs  in  part  with  this  recommendation.  LJC  has  established 
standard  Rating  Definition  Levels  (RDLs)  for  fielded  force  organizations.  All 
partners,  mentors  and  commanders  are  being  trained  on  these  RDLs.  LJC 
assessments  personnel  have  coordinated  with  and  are  in  routine  contact  with 
NTM-A  assessments  personnel  to  ensure  full  understanding  of  our  respective 
systems.  LJC  RDLs  and  the  standards  associated  with  them  may  not  be 
appropriate  for  Ministerial  Level  assessments  conducted  by  NTM-A/ CSTC-A. 


1 
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SeeSIGAR 
comment  1. 


SeeSIGAR 
comment  2. 


RECOMMENDATION  5.  (page  19.  SIGAR  Draft) 

Incorporate  present-for-duty  figures  as  a  percentage  of  assigned  personnel  into 
assessments  for  both  police  ana  army  units.  Note:  Until  reliable  present-for-duty  numbers 
are  available  for  police  units,  we  recommend  that  DC  incorporate  a  separate  measurement 
for  tracking  "available"  personnel  who  have  been  detailed  to  other  locations  (i.e.,  available, 
but  not  present). 

DC  RESPONSE:  UC  concurs  with  and  has  implemented  this  recommendation.  The 
new  CUAT  collects  present  for  duty'  information  for  both  police  and  army 
formations.  Note:  Because  not  all  police  entities  are  partnered,  UC  plans  to  take 
those  units  that  are  reported  and  determine  an  average  present  for  duty' 
percentage.  This  average  present  for  duty  will  be  reported  until  all  police  units  are 
partnered  and  reporting  present  for  duty'  figures. 

RECOMMENDATION  6.  (page  19.  SIGAR  Draft) 

Measure  the  ratios  of  trained  and  ce-tfied  drivers  to  vehrdes  on  hand. 

DC  RESPONSE:  UC  does  not  concur  with  this  recommendation.  Although  this 
specific  issue  is  of  concern  at  the  tactical  level,  it  is  one  of  many  such  issues. 
Highlighting  each  of  them  by  separate  reporting  fields  would  offer  little  value  in 
understanding  combat  effectiveness  across  the  force.  Issues  such  as  shortage  of 
drivers  can  be  (and  are  being)  addressed  in  the  appropriate  RDLs  for  manning, 
logistics,  etc.,  and  can  be  brought  to  the  attention  of  the  appropriate-level 
commander  through  this  means. 

RECOMMENDATION  7.  fpape  19.  SIGAR  Draftl 

Conduct  independent  val  idations  of  top-level  Afghan  police  unit  capabilities. 

DC  RESPONSE.  UC  concurs  with  and  is  exploring  options  to  implement  this 
recommendation.  IJC  is  reviewing  the  mission,  manning,  and  training  of  the 
existing  Validation  Training  Team  organization  to  determine  their  suitability  for 
conducting  independent  assessments  of  top-level  Afghan  Police  and  Arnyy 
organizations. 

To  improve  the  transparency  and  accuracy  of  ANSF  assessment  reporting,  we 

recommend  tnat  DC: 

RECOMMENDATION  8.  (page  19.  SIGAR  Draft) 

Indicate  the  number  cf  months  since  the  last  direct  assessment  when  carrying  over  ratings, 

IJC  RESPONSE:  UC  does  not  concur  with  this  recomnyendation.  The  IJC  does  not 
intend  to  carry'  over  assessments  from  one  period  to  the  next.  The  unit  either  has 
a  current  assessment  or  will  be  reported  as  not  assessed.  Carrying  over  ratings 
could  have  a  misleading  effect  on  the  overall  assessnyent  process  and  could  result 
in  units  feeling  they  do  not  have  to  maintain  current  assessnyents.  In  cases  where 
a  "not  assessed"  rating  is  assigned,  the  tinye  elapsed  since  the  last  valid 
assessnyent  will  be  readily  apparent  in  that  unit's  data  record  over  time. 
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RECOMMENDATION  9.  (page  19.  SIGAR  Draft) 

Begin  to  systematically  track  the  number  of  unit-level  evaluations  not  submitted  each 
month  ana  the  number  of  elements  not  observed  within  those  assessments. 

DC  RESPONSE:  DC  concurs  with  and  will  implement  this  recomntendabon.  The 
CUAT  systematically  tracks  the  number  of  unit-level  evaluations  not  submitted 
each  ntonth  and  the  nuntber  of  elements  not  assessed  during  each  reporting 
period. 

RECOMMENDATION  10.  (page  20.  SIGAR  Draft) 

Add  an  additional  overall  rating  level  to  represent  units  "not  assessed’  where  no 
mentor/partners  are  assigned. 

DC  RESPONSE:  DC  partially  concurs  with  this  recomn>endation.  The  CUAT 
identifies  units  that  were  not  evaluated  and  should  have  been ,  regardless  of 
partnering  status.  The  CUAT  also  identifies  units  that  are  part  of  the  ANSF 
organizational  structure  but  were  not  evaluated  because  they  have  not  yet  been 
fielded.  We  will  examine  the  possibility'  and  benefit  of  adding  a  mechanism  for 
showing  a  unit  was  not  evaluated  because  there  was  no  mentor/ partner. 

To  maximize  visibility  into  Afghan  police  capabilities,  we  recommend  that  DC: 

RECOMMENDATION  11.  (page  20.  SIGAR  Draft) 

Require  all  U.S.  and  Coalition  military  mentor  and  partner  teams  to  submit  unit  assessments 
and  that  DC  begin  requesting  unit  assessments  from  avilian  advisers  in  those  districts 
where  military  mentors/partners  are  not  assigned. 

DC  RESPONSE:  DC  partially  concurs  with  this  recommendation.  DC-ADAB  does 
not  have  the  authority  to  require  all  US  and  Coalition  ntentors  and  partner  teams 
to  submit  unit  assessments  to  the  DC.  Some  teams  are  under  contract  from  the  US 
State  Department  and  some  are  partnered  based  on  bi-lateral  arrangements 
between  the  mentor  nation  and  the  Afghan  government.  DC-ADAB  does  request 
assessments  from  these  mentors  and  some  have  provided  valuable  insight  but 
there  is  no  standardized  format.  DC-ADAB  will  attempt  to  explain  the  CUAT 
process  to  all  mentor  and  partner  organizations  and  encourage  their  use  of  this 
standardized  process  but  we  currendy  lack  authority  to  ensure  compliance  in  all 
cases. 


To  counteract  perverse  incentives  resulting  from  the  application  of  the  ANSF 
assessment  system,  we  recommend  that  DC: 

RECOMMENDATION  12.  (page  2Q,  SIGAR  Draft) 

Evaluate,  in  conjunction  with  NTM-A^CSTC-A,  MOI,  and  MOD,  the  creation  of  an  incentive 
system  to  rewara  Afghan  units  for  the  development  of  validated,  top-level  operational 
capabilities. 

IJC  RESPONSE;  DC  concurs  with  this  recontmendation.  DC  will  in  coordination 
with  GIRoA  partners  and  I  SAP  HQ  the  feasibility  of  implementing  a  positive 
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incentive  program  which  rewards  ANSF  organizations  that  achieve  top  level 
operational  effectiveness  and  transition  to  self-sufficient  operations. 

GENERAL  COMMENTS  ON  THE  REPORT 

Broadly  speaking,  the  recommendations  in  this  report  reflect  actions  that  were 
already  in  progress  when  the  SIGAR  audit  team  visited  UC  HQ,  and  upon  which 
the  team  was  briefed  during  their  visit  It  is  reassuring  that  the  SIGAR  audit  team 
has  indicated  positive  support  to  such  a  wide  range  of  UC  initiatives.  The  report 
should  have  great  utility  in  bringing  these  actions  to  the  attention  of  a  broader 
audience. 

The  general  picture  painted  in  the  report  regarding  the  challenges  to  assessment 
of  operational  effectiveness  of  ANSF  organizations,  and  the  majority  of  the  specific 
comments  regarding  the  now- superseded  TRAT/CE  system,  are  accurate  from  the 
UC's  perspective. 


APPROVED  BV: 

D.  M.  Rodriguez 
LTG,  U.S.  Army 
Commander,  UC 


PREPARED  BY: 

M.  L.  Jones 

LtCd,  U.S.  Marine  Corps 
HQ  DC  DSN  (318)  449-9032 
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The  following  are  SIGAR's  comments  on  IJC's  letter  dated  June  15,  2010: 

1.  We  removed  this  recommendation  from  our  final  report,  as  the  actions  IJC  described  in  its 
response  fulfilled  the  spirit  of  our  recommendation. 

2.  We  removed  this  recommendation  from  our  final  report,  as  IJC's  description  of  its  intentions 
fulfilled  the  spirit  of  our  recommendation. 
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APPENDIX  III:  COMMENTS  FROM  NATO  TRAINING  MISSION -AFGHANISTAN/COMBINED 
SECURITY  TRANSITION  COMMAND-AFGHANISTAN 


SeeSIGAR 
comment  1. 


SeeSIGAR 
comment  2. 


SeeSIGAR 
comment  3. 


HEADQUARTERS 

NATO  TRAINING  MISSION  •  AFGHANISTAN 
COMBINED  SECURITY  TRANSITION  COMMAND  •  AFGHANISTAN 
KABUL  AFGHANISTAN 
APO  AE  CS356 


MUrro 
ATTENTION  OF 


NTM-A/CSTC-A-CG 


MEMORANDUM  THRU 


10  June  2010 


Commander,  United  States  Forces  -  Afghanistan.  APO  AE  09356 
Commander,  United  States  Central  Command,  MacDill  AFB.  FL  33621 

FOR  Office  of  the  Special  Inspector  General  for  Afghanistan  Reconstruction.  400  Army-Navy 
Drive,  Arlington,  VA  22202 

SUBJECT:  NTM-A  Response  to  Draft  S1GAR  Audit  Report  dated  June  2010 


1 .  The  purpose  of  this  memo  is  to  highlight  the  fact  that  the  information  in  the  Draft  SIGAR 
Audit  Report,  dated  June  2010,  is  based  on  data  that  is  six  months  old.  The  findings  in  this 
report  do  not  accurately  represent  the  current  state  of  the  Afghan  National  Security  Force 
(ANSF)  training,  or  recognize  the  progress  the  Afghan  national  security  leadership  have  made  in 
reversing  the  adverse  trends  in  the  growth  and  professionalization  of  the  ANSF  that  existed  just 
six  months  ago.  In  light  of  the  tremendous  progress  made,  while  still  recognizing  the  significant 
challenges  remaining,  this  report  is  not  only  inaccurate,  it  is  potentially  damaging. 

2.  When  NTM-A  activated  six  months  ago,  the  command  faced  several  critical  challenges:  some 
training  facilities  were  on  the  verge  of  shutting  down  due  to  severe  Coalition  manning  shortages; 
the  training  focus  was  on  quantity  over  quality;  most  Afghan  National  Police  ( ANP)  were 
recruited  and  assigned,  but  seldom  received  any  formal  training;  and  ANSF  recruiting  and 
retention  w  as  lagging  partially  due  to  the  inequality  of  pay  in  comparison  to  similar  occupations. 
In  the  past  six  months,  several  substantial  changes  in  the  approach  to  training  not  only  turned 
around  the  training  effort  but  also  created  a  sense  of  urgency  to  build  a  foundation  this  year  for 
an  enduring,  professional  ANSF.  Changes  in  systems,  procedures,  and  culture  began 
immediately,  but  these  take  time  for  demonstrable  progress  to  be  realized.  While  the  SIGAR 
audit  was  done  over  this  period,  the  report  does  not  capture  the  significant  radically  new 
processes  which  have  led  to  the  currently  demonstrated  progress.  Discussing  this  new  approach 
and  the  recently  emerging  trends  will  provide  a  more  accurate  context  for  those  reading  the 
report.  It  is  imperative  that  the  SIGAR  report  portray  information  in  an  accurate  and  timely 
manner,  which  unfortunately  has  not  been  done.  A  more  accurate  portrayal  of  the  current 
situation  and  challenges  ahead  can  be  found  in  the  attached  ‘NTM-A  1 80  Day  Internal 
Assessment  and  W'ay  Forward.'  (supporting  data  slides  are  available  upon  request). 

3.  Additionally,  the  report  is  critical  of  the  Capability  Milestone  (CM)  rating  system.  The 
command  understands  the  SIGAR  position  that  the  CM  rating  system  is  not  perfect  in  measuring 
the  development  of  ANSF  capabilities  against  end-state  goals  -  no  one  claimed  this  as  their 
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SeeSIGAR 
comment  3. 


NTM-A/CST  C-A 
Response 

Draft  SIGAR  Audit  Report  No.  10-1 1 

Actions  Needed  to  Improve  the  Reliability  of  Afghan  Security  Force  Assessments 

May  2010 

desired  outcome.  In  fact,  the  October  2009  DoD  Report  to  Congress  on  the  Progress  Towards 
Security  and  Stability  in  Afghanistan  clearly  states  that: 

"It  is  important  to  understand  fully  what  CM  ratings  measure  and  indicate.  CM  ratings 
simply  depict  the  manning,  training,  and  equipment  of  a  unit.  The  correlation  between 
CM  ratings  and  operational  capability  to  complete  assigned  missions  is  tenuous,  and  thus 
attempting  to  draw  the  conclusion  that  a  CM  rating  is  an  indicator  of  the  capacity  for 
success  in  operations  can  be  misleading.*’ 

In  recognition  of  the  failure  of  CM  ratings  to  assess  operational  capability,  NTM-A/CSTC-A  and 
1JC  began  in  the  fall  of  2009  to  reform  the  system  to  provide  a  better  assessment  of  ANSI' 
progress. 

4.  Attached  are  our  comments  on  the  draft  SIGAR  Report  and  our  1 80  Day  Internal  Review  and 
Way  Forward  which  together  (1)  provides  some  historical  context  regarding  the  challenges  wc 
have  collectively  faced  as  we  build  the  ANSF  in  a  counterinsurgency  environment;  (2)  provides 
a  sense  of  balance  to  the  anecdotal  and  dated  observations  contained  within  die  draft  SIGAR 
report;  (3)  highlights  initiatives  that  have  been  instituted  and  clearly  illustrate  the  significant 
progress  that  has  been  achieved  since  the  research  was  conducted  for  this  audit,  as  well  as  our 
projections  for  future  growth  and  the  significant  challenges  ahead;  (4)  responds  to  specific 
observations,  findings,  and  comments  within  the  draft  report;  and  (5)  responds  to  the  three 
recommendations  at  the  end  of  this  draft  report. 

5.  Point  of  contact  for  this  action  is  COL  Jeffrey  L.  Kent,  Senior  ANSF  IG  Advisor/Command 
IG  at  DSN  (318)  237-1076,  email:  icfffcv  l.kcnt  a  aafghan.swa  annv.mil. 


WILLIAM  B.  CALDW’ELL.  IV 
Lieutenant  General,  USA 
Commanding 


2  Ends; 

1.  NTM-A'CSTC-A  Responses.  10  Jun  10 

2.  1 80  Day  Internal  Review  and  Way  Forward,  29  May  10 
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Page  14,  para  2:  “DC  and  NTM-A/CSTC-A  officials  said  that  mentor/partner  performance  had 
a  significant  effect  on  development  and  assessment  practices.  Anecdotal  evidence  pointed  to 
variation  in  mentor/partner  efforts,  but  DC  and  NTM-A/CSTC-A  did  not  attempt  to  assess 
performance  systematically,  according  to  these  officials  .” 

NTM-A/CSTC-A  Response:  NTM-A/CSTC-A  believes  that  "mentor, 'partner  performance  had 
a  significant  effect  on  development  and  assessment  practices.” 

Page  17,  para  1:  “Mentors  and  partners  acknowledged  that  widespread  efforts  have  been  under 
way  by  the  United  States  and  Coalition  partners  to  help  improve  ANSF  facilities.  Nevertheless, 
they  also  reported  pervasive  delays  in  construction  projects". 

NTM-A/CSTC-A  Response:  This  draft  report  also  references  the  SIGAR  audit  on  the  ANA 
Garrison  at  Kunduz.  which  they  mention  “persistent  delays"  It  is  important  to  note  that  the 
Kunduz  project  was  a  "fixed -firm  priced"  contract,  and  this  section  infers  that  there  were 
construction  cost  over-runs,  which  was  not  the  case.  We  exercised  contract  options  due  to 
accelerated  growth  of  the  ANA.  The  exercise  of  contract  options  routinely  impacts  a  contract's 
completion  schedule  and  die  contract's  price.  Consequently,  quite  frequently  construction 
contracts  are  not  completed  within  the  original  schedule,  since  the  original  schedule  is 
superseded  due  to  exercising  on  option(s). 

Page  1 7,  para  3:  Corruption  has  undermined  the  development  of  independent  ANSF 

NTM-A/CSTC-A  Response:  1SAF  and  NTM-A’CSTC-A  is  well  aware  of  corruption  within  the 
Afghan  government  and  ANSF.  Corruption  is  a  difficult  and  inherent  problem  in  this 
environment.  However,  both  ISAF  and  NTM-A/CSTC-A  arc  taking  actions  to  combat 
corruption.  MOl's  conduct  of  the  Personnel  Asset  Inventory’  (PAI)  was  their  first  major  step  in 
reducing  corruption  in  the  police  force.  Establishing  a  dataha.se  of  all  ANPs  allows  MOI  and 
NTM-A  to  eliminate  double  enrolled  police  and  accompanying  corruption,  and  allow  for  a  more 
transparent  organization.  The  PAI  will  provide  a  baseline  for  the  police  force  to  maintain 
accountability  and  eliminate  corruption  within  the  ranks.  The  process  includes  registration,  drug¬ 
testing.  re-vetting,  weapons  verification  and  obtaining  biometrics  information  for  100%  of  the 
ANP. 

Measures  such  as  pay  parity  between  police  and  the  army,  pay  delivery,  and  pay  accountability 
axe  a  strong  anti-corruption  measure  that  has  been  implemented  across  the  ANSF.  llic 
implementation  of  Electronic  Funds  Transfer,  which  pays  personnel  by  direct  deposit  into  bank 
accounts  rather  than  cash  payments,  continues.  Since  one  of  the  obstacles  to  across  the  board 
implementation  of  an  EFT  system  is  the  lack  of  banks  or  ATMs  in  rural  areas  of  Afghanistan,  the 
testing  of  an  alternative  new  technology  has  begun  in  select  ANP  districts.  This  cell  phone 
technology  ’Roshan  m-Paisa"  will  provide  a  method  for  ANA  and  ANP  soldiers  to  receive  a 
secure  PIN  code  on  their  cell  phone  that  allows  them  to  draw  their  pay.  These  two  methods 
streamline  direct  pay  and  account  for  78%  of  all  pay  accounts,  increasing  monthly. 
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Page  18,  para  3:  “Drug  abuse  continues  to  be  a  problem  for  the  Afghan  army  and  police  As  of 
February  2010,  results  from  drug  test  during  a  personnel  asset  inventory  of  the  Afghun  police 
force  overall  showed  positive  results  for  drug  use  in  17  percent  of  police  tested.  ' 

NTM-A/CSTC-A  Response:  NTM-  j VC  S  TC  -  A  is  aware  that  drug  abuse  is  a  problem. 

However,  they  arc  working  with  the  MOI  to  correct  this  problem.  The  MO!  has  developed  a  4 
pillar  program  of  awareness,  identi fication.' testing.  enforcement,  and  referral  for  treatment. 
During  the  recent  PA1.  MOI  conducted  testing  of  97.000  personnel  and  continue  testing  in 
training  centers  with  a  goal  of  1 0(1%  tested  by  2**  Qtr  F Y 1 0.  Mandatory  drug  awareness 
modules  are  being  incorporated  into  courses  at  regional  training  centers  (RTCs).  MOI  is 
assigning  Islamic  religious  leaders  to  provinces  to  address  the  religious  implications  of  drug  use. 
Finally,  we  are  soliciting  contributions  from  the  International  Community  for  ANP 
Rehabilitation  Facility  under  Ministry  of  Public  Health. 

Page  18,  para  4:  "Although  steps  have  been  token  to  introduce  literacy  training  for  the  Afghan 
army  and  police,  high  levels  of  illiteracy  in  both  forces  continue  to  hamper  development.'' 

NTM-A/CSTC-A  Response:  Afghanistan's  illiteracy  problem  did  not  occur  overnight,  nor  will 
it  be  solved  overnight.  T  his  is  a  well  known  problem  and  likely  will  be  for  quite  some  time,  but 
NTM-A/CSTC-A  has  initiatives  to  combat  illiteracy.  In  past  progiams,  literacy  training  had  been 
voluntary,  now  we  have  made  it  mandatory  for  all  ANSI-  soldiers  and  police.  In  addition,  the 
number  of  literacy  instructors  has  increased  from  251  in  November  2009  to  351  in  May  2010, 
and  is  projected  to  have  1,011  instructors  by  November  2010.  The  number  of  students  has  also 
increased  from  6,345  in  November  2009  to  9,191  in  May  2010.  NTM-A/CST-A  projects  there 
will  be  28.359  enrollments  by  November  2010.  As  the  number  of  students  is  increasing,  the 
number  graduates  is  also  increasing.  The  six  months  beginning  in  November  2009  to  May  2010 
was  extremely  successful,  going  from  0  graduates  at  the  beginning  of  the  period  to  1 ,789  by 
periods  end.  The  next  six  months  are  projected  to  be  even  better,  with  a  projected  10,167 
graduates  by  November  2010. 

SIGAR  specific  recommendations  for  NTM-A/CSTC-A: 

1 .  Work  with  MOI  and  MOD  to  implement  functional  tracking  systems  for  monitoring  supply 
requests. 

NTM-A/CSTC-A  Response:  Concurs  and  has  taken  steps  to  address  this 
recommendation.  This  issue  is  of  great  concern  to  NTM-A/CSTC-A  and  rectify  ing  the 
missing  components  in  the  ANSF  logistics  system  will  lead  to  greater  trust,  less 
confusion,  and  enhanced  operational  readiness  throughout  the  ANSF.  These  issues, 
with  others,  were  previously  identified  and  are  programmed  into  the  next  ANSF 
logistics  decree  review,  update,  and  rewrite. 

Although  the  establishment  of  the  FSDs  and  RLCs  has  greatly  increased  the  capability 
of  the  ANSI's  logistics  operations,  the  existing  ANSF  logistics  configuration  has 
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multiple  layers  of  bureaucracy  hamper  efficiency  and  effectiveness.  For  example  a 
supply  request  require  a  minimum  of  seven  signatures:  due-ins  and  due-outs  are  only 
kept  manually  at  stock  record  points;  and  the  priority  for  materiel  is  new  units  which 
are  issued  to  a  level  of  70%  to  80%,  due  to  atuition,  before  declaring  them  mission 
capable.  Incomplete  information  in  an  undermanned  bureaucratic  system  of  scarce 
resources  has  resulted  in  turmoil,  workarounds,  consternation,  and  readiness  issues. 

NTM-A/CSTC-A  has  had  to  implement  or  accelerate  policies  and  procedures  to  case 
the  grief.  One  of  the  policies  is  the  “fill-or-kill  policy”  for  supply  requests  for  new 
units.  If  the  materiel  is  not  available  the  request  is  canceled  for  the  shortage.  This  is 
expected  to  eliminate  multiple  requests  for  the  same  item  when  status  is  unavailable. 

NTM-A/CSTC-A  has  started  to  field  the  Authorized  Stockage  Listing  (ASL)  for 
regional  supply  points  tha!  have  been  completed  so  as  to  provide  materiel  management 
capability  at  the  local  level  rather  than  at  the  national  level  in  Kabul.  Another 
component  in  rectify  ing  the  problems  is  that  NTM-A/CSTC  will  be  fielding  and 
developing  additional  ANSI  logistics  enablers  beginning  FY 1 1 .  Finally,  the 
accelerated  growth  and  disparity  of  logistics  maturity  between  ANA  and  ANP  has  put  a 
burden  on  the  supply  system,  where  item  lead  times  can  be  in  the  "years”,  causing  new 
ANSF  units  and  fielded  units  to  compete  for  the  same  scarce  resources.  liven  though  it 
is  a  conscience  US  policy  decision  to  develop  an  Afghan  fighting  force  before  a 
logistics  force,  NTM-A/CSTC-A  is  working  with  the  ANSI-'  to  develop  a  reliable 
logistics  system  that  the  ANSF  trusts. 

2.  Develop  a  program  to  systematically  provide  backfill  training  for  new'  recruits  assigned  to 
previously-trained  FPDD  districts. 

NTM-A/CSTC-A  Response:  Concurs  and  has  taken  steps  to  address  this 
recommendation.  Prior  to  November  2009  most  ANP  were  recruited  and  assigned,  but 
seldom  received  any  formal  training.  This  model  -  Recruit-Assign-Intend  to  Train  - 
was  replaced  with  a  new  model  that  makes  training  mandatory  for  all  police  recruits. 
We  anticipale  that  this  new  model,  described  as  Recruit-  Train-Assign,  will  improve 
quality  .  We  have  also  instituted  in  the  training  base  increasing  numbers  of  specialized 
courses  to  develop  quality  skill  sets  necessary-  to  support  the  fielded  force.  Recruit- 
Train-Assign  will  mitigate  the  decline  in  capability  of  FPDD  districts  as  they  lose 
police  to  attrition,  since  all  incoming  recruits  will  have  received  formal  training  prior  to 
iheir  arrival  at  their  units.  This  model  will  eliminate  the  requirement  for  reform  and  1JC 
is  considering  phasing  out  FPDD  after  FPDD  12.  Also.  MOI  has  developed  the  new 
ANP  Training  Command  designed  to  institutionalize  training  and  bring  unity  of  effort 
to  the  .-ANP  training  mission. 

3.  Implement  a  train-the-trainer  driving  program  to  allow  each  ANSF  unit  to  extend  driver 
training  using  its  own  personnel. 
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NTM-A/CSTC-A  Response:  Concurs  and  has  taken  step®  to  address  this 
recommendation.  Both  the  ANA  and  the  ANP  have  implemented  drivers  training 
programs  in  their  training  centers.  The  ANA  conducts  driver’s  familiarization  at  the 
Consolidated  Fielding  Center  in  order  to  familiarize  ANA  soldiers  with  the  vehicles 
assigned  to  their  Kandak.  Additionally,  a  five  week  driver's  training  course  is  taught  at 
the  logistics  school.  The  ANA  Logistics  school  also  conducts  a  driving  instructors 
course  and  driving  basic  course  which  they  have  graduated  1 1 3  and  23 1  personnel 
respectively.  The  ANP  are  taught  to  drive  both  the  light  pick-ups  they  are  issued,  as 
well  as  a  smaller  number  arc  trained  to  drive  the  UAH’s,  that  are  now  being  fielded  to 
ANP  units.  To  date,  the  ANP  has  trained  986  dnvers  through  the  UAH  driver's  training 
course.  A  consistent  trend  seen  across  the  board  is  the  very  small  number  of  recruits 
that  join  the  army  or  police  with  the  existing  knowledge  of  how  to  drive. 


APPROVED  BY: 

WILLIAM  B  CALDWELL,  IV 
Lieutenant  General.  USA 
Commanding 


PREPARED  BY: 
JEFFREY  L  KENT 
COL,  USA 
Command  IG 
N 1  M-AC5TC-A 
<318)237-1076 
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29  May  2010 


ST  "RTF CT:  180  Day  Trtcmal  Review  and  Way  Forward 

1.  Allow  roe  to  convey  on  behalf  of  the  Afghan  People  and  the  International  Community  our 
sincerest  thanks.  You  have  dorse  a  truly  incredible  job  in  a  very  short  period  of  rime.  Your 
tireless  efforts  over  the  last  six  months  in  working  closely  with  our  Afghan  partners  forged  a 
comprehensive  approach,  underpinning  what  wc  collectively  achieved.  Each  of  you  should  be 
extremely  proud  of  what  you  and  our  Afghan  partners  have  accomplished — laying  the 
foundation  for  an  enduring  legacy  of  security.  Am  proud  to  be  serving  with  each  of  you  and 
look  forward  to  what  we  will  accomplish  as  wc  work  with  our  Afghan  partners  as  one  team. 

2.  Overview.  In  the  first  six  months  oFNTM-A,  we  changed  our  approach  to  training  and 
created  a  seme  of  urgency  as  we  build  a  foundation  for  an  enduring,  professional  ANS1;  NATO 
and  the  Afghan  security  leadership  made  significant  progress  in  reversing  adverse  trends  in  the 
growth  and  professionalization  of  the  ANSF  However,  wc  still  have  much  to  do.  Building  an 
enduring  and  self-sustaining  force  remains  a  distinct  challenge  ar.d  attainment  of  the  growth 
objectives  is  not  assured.  While  aggregate  growth  objectives  appear  on  track,  higher  tempo 
operations  combined  with  the  onset  of  the  traditional  summer  lull  (Jul-Oct)  in  recruiting  will 
significantly  challenge  our  programs,  both  in  regard  to  quantity  and  quality.  The  development  of 
effective  leaders  and  the  persistent  high  attrition  in  specific  components  of  the  ANSF  also  pose 
major  challenges.  In  looking  forward,  the  probability  of  successful  growth  and  development  of 
the  ANSF  can  be  improved  if  our  programs  receive  full  Coalition  and  Afghan  instructor 
manning,  and  if  requirements  of  longer  term  ANSF  growth  and  professionalization  are  carefully 
balanced  against  current  operational  demands  in  the  field.  To  provide  perspective,  the  review 
that  follows  will  describe  conditions  at  activation  of  NTM-A,  detail  the  conditions  at  the  time  of 
writing  this  document,  and  explain  in  more  detail  our  challenges  and  concerns  as  we  move 
forward. 

3.  Conditions  at  Establishment  of  NTM-A,  When  NTM-A  activated  we  faced  several  urgent 
challenges  Fitst.  some  training  facilities  were  on  the  verge  of  shutting  down  due  to  severe 
Coalition  manning  shortages.  In  November  2009,  NTM-A/CS  I'C-A  was  manned  at  little  more 
than  25%,  and  the  trainer-to-trainee  ratio  for  the  Afghan  National  Army  (ANA)  was 
approximately  1 :79.  although  the  ratio  was  far  worse  (as  low  as  1  4f>6)  at  some  training 
locations.  Second,  the  training  focus  was  on  quantity'  over  quality;  there  were  in  fact  Tew.  if  any, 
quality  standards  such  that  all  trainees  present  on  graduation  day  typically  graduated,  and  the 
marksmanship  pass  rate  hovered  at  35%.  Third,  most  Afghan  National  Police  (AN'P)  were 
recruited  and  assigned,  but  seldom  received  ary  formal  training.  Fourth,  ANSF  pay  lacked 
parity  with  comparable  occupations,  exerting  a  negative  impact  on  recruiting  and  retention,  and 
increased  attrition.  Finally,  overall  recruiting  in  the  Fall  of  2009  had  dipped  to  near  record  lows. 

4.  Six  Month  Review.  In  the  last  six  months  and  as  International  Security  Assistance  Force’s 
strategic  main  effort,  we  have  accomplished  much  to  stabilize  the  Afghan  training  mission. 
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Recently  consolidated  metrics  capture  the  change  in  performance  of  NTM-A/CSTC-A  programs 
between  November  2009.  now,  and  where  we  project  them  to  be  in  November  201 0.  Most  of 
our  indicators  are  focused  on  inputs;  however,  we  will  work  with  the  fielded  foroc  to  better 
capture  and  measure  outputs  and  outcomes. 

a  NTM-A/CSTC-A  Manning.  Our  Coalition  manning  improved  to  70%  overall  and  the 
instructor-to-traincc  ratio  to  1 :29,  but  both  remain  significant  concerns.  In  order  to  increase  the 
probability  of  success  and  increase  the  quality  o'  training,  additional  personnel  arc  required.  The 
U.S.  provided  bridging  solutions,  but  NATO  partners  must  begin  to  follow  through  on  their 
pledges  by  late  summer  to  have  an  impact  this  year.  Manning  shortages  have  an  overall  negative 
impact  on  training  operations,  with  institutional  trainer  shortages  at  police  training  centers  and 
ANA  branch  schools  being  the  most  serious. 

b.  ANSF  Quantity.  Training  facilities  have  been  expanded  and  throughput  increased  such 
that  ANSF  is  currently  on  path  to  meet  the  end  strength  objectives  of  1 Q9K  for  .ANP  and  134K 
for  ANA  by  31  Oct  2010  However,  component  elements  of  the  ANA  and  ANP  are  at  risk  to 
meet  their  end  strength  goals,  such  as  ANCOP  (Afghan  National  Civil  Order  Police),  and  ANA 
units  thal  have  a  high  operational  tempo  may  experience  rising  attrition.  I  be  traditional  summer 
lull  for  ANSF  recruiting  exacerbates  quantity  concerns. 

(1)  Recruiting  Mitigation.  Faced  with  low  recruiting  in  late  2009.  it  was  apparent 
we  would  not  make  growth  objectives.  We  worked  together  with  Afghan  leadership  to 
implement  several  initiatives  to  mitigate  low  recruiting,  to  include  pay  raises,  taping  the  number 
of  recruiters,  and  stand  up  of  a  Recruiting  Command  En  addition,  we  contracted  for  a  major 
Afghan  media  recruiting  campaign,  authorized  a  further  increase  in  recruiters,  and  set  up  sixteen 
mobile  sub-recruiting  stations  to  coincide  with  the  start  of  the  challenging  summer  months. 

(2)  Attrition  Mitigation.  Aggregate  attrition  has  been  decreasing  for  the  ANA  and 
ANT  over  the  last  six  months,  but  there  is  no  assurance  thal  this  trend  will  continue,  especially  in 
those  ANA  and  ANCOP  units  engaged  in  high  tempo  operations.  Despite  its  small  size.  ANCOP 
is  of  particular  concern  given  its  key  role  in  counterinsurgency  operations  and  its  proportionately 
large  share  of  the  overall  ANT  growth  program.  Wliile  down  from  highs  of  November  2009,  the 
ANCOP  level  of  attrition  remains  unacceptable  and  unsustainable  at  an  approximate  annualized 
rate  of  70%  (as  determined  by  most  recent  measures  taken  in  April  of  this  year).  To  mitigate  the 
effects  of  high  ANCOP  attrition,  recruiting  has  been  ramped  up.  Additionally,  we  arc  working 
closely  with  the  fielded  force  to  reduce  attrition  through  several  initiatives. 

c.  ANSF  Oualitv.  Several  NTM-A  programs  have  been  reoriented  to  place  a  greater 
emphasis  on  quality,  and  progress  has  been  noted  in  some  areas. 

(1)  Training  Programs.  We  focused  early  on  marksmanship,  raising  qualification 
rates  in  ANA  from  35%  to  65%  by  May.  and  expanded  driver  training.  Up  until  March  of  this 
year,  most  AUP  were  recruited  and  assigned  without  formal  training  This  model — Recruit- 
Assign-Intend  to  Train — was  replaced  with  a  new  model  thal  makes  training  mandatory  for  all 
police  recruits.  Wc  anticipate  that  this  new  model,  described  as  Recruit-Train-Assign,  will 
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improve  quality.  We  have  also  instituted  in  the  training  base  increasing  numbers  of  specialized 
courses  to  develop  quality  skill  sets  necessary  to  support  the  fielded  force 

(2)  Literacy.  At  the  Stand-up  of  NTM-A.  there  existed  minimal  mitigation 
programs  to  improve  the  quality  of  the  force,  and  these  were  voluntary.  In  the  past  six  months 
we  have  significantly  expanded  and  made  mandatory  across  ANSF  bask  literacy  programs.  Wre 
see  literacy  programs  as  a  major  quality  enhancement  tool  but  also  an  incentive  for  recruiting 
and  possibly  retention. 

(3)  Leadership.  ANA  and  ANP  leader  development  courses  have  been  developed, 
reorganized,  and  unproved  to  support  increased  through-put  requirements  of  the  Gelded  force. 
Steps  have  also  been  taken  to  ensure  higher  quality  and  highly  literate  officers  are  distributed 
hased  on  operational  need,  not  cronyism  or  favoritism.  To  that  end.  the  National  Military 
Academy  of  Afghanistan  (N'MAA)  instituted  a  transparent  assignment  lottery  sy  stem  that 
distributed  graduates  of  the  class  of  2010  to  all  the  fielded  force,  rather  titan  predominantly  in 
Kabul,  which  had  been  the  case  in  the  2009  graduating  class. 

(4)  Intelligence.  While  the  initial  cadre  is  relatively  small,  we  have  established  the 
Personnel  Security  (PERM EC)  vetting  program  for  the  Ministry  of  Defense  (MoDi  that  will 
eventually  expand  to  be  ANSF-wide,  to  include  the  Ministry  of  Interior  (Mol)  in  June  of  this 
year.  With  regard  to  improving  the  quality'  and  capacity  of  ANSF  intelligence  programs,  we 
assisted  in  the  establishment  of  a  nctwotk  targeting  and  exploitation  center,  which  will  help  the 
ANP  combat  insurgent  and  criminal  networks,  and  thus  complement  recent  new  intelligence 
training  programs  that  are  already  paying  dividends. 

(5 )  Sustainment.  We  have  taken  additional  steps  to  improve  the  logistical  and 
mcdkal  sustainment  capacity  of  ANSF.  examples  of  which  include  the  standup  of  Regional 
Logistic  Centers  (RIjCX  a  Medical  Training  Advisory  Group  (MTAG),  and  the  establishment  of 
educational  and  clinical  standards  for  doctors.  However.  ANSF  sustainment  capacity  remains  a 
challenge  due  to  the  shortage  ol  trained  Afghan  logistic  personnel. 

(6)  Air  Corps  Another  early  sign  of  increasing  quality  efforts  was  demonstrated 
in  the  expansion  of  the  Air  Corps,  a  highly  technical  community  that  must  maintain  high  quality 
standards  in  both  training  and  maintenance  in  order  to  safely  operate.  To  that  end,  Afghan  air 
capability  was  successfully  expanded  with  the  introduction  of  the  first  live  of  twenty  planned  C- 
27a,  the  first  modem  airframe  in  the  Air  Corps’  inventory.  The  Air  Corps  established  an  Air 
School  for  education  and  training,  in  addition  to  increased  capabilities  in  battlefield  mobility, 
casualty  evacuation,  forward  observer  training,  humanitarian  support,  and  mission  planning. 

d.  Ministerial  Development  Ministerial  development  shows  slow  but  steady  incremental 
progress,  an  improvement  when  compared  to  the  low  level  of  systems  maturity  in  early  2009. 
The  MoD  and  Mol  arc  moving  towards  Afghan-led  ministerial  operational  capacity,  but  neither 
will  be  fully  capable  before  2012.  A  lack  of  quality  leaders,  middle-level  staff,  and  efficient 
bureaucracy,  however,  pose  challenges  which  wc  will  mitigate  through  training  and  education 
programs,  and  experience  gained  by  close  partnering  with  CSTC-A  advisors. 
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c.  Anti-Corruption  Efforts.  Our  anti  corruption  philosophy  is  not  limited  to  prosecution 
only,  but  also  seeks  lo  promote  policies  and  cultures  that  engender  transparent  and  accountable 
practices.  Important  components  of  this  strategy  include  “Pay  by  Phone”.  Electronic  Funds 
Transfer,  and  Mol  mobile  anti -corruption  teams  which  have  successfully  conducted  several 
investigations.  Additionally,  the  establishment  of  an  ANSF  biometrics  program,  the 
development  of  merit-based  promotion  policies,  and  the  N'MAA  loitcry  are  all  contributing  to  an 
environment  of  transparency. 

5.  Looking  Forward:  Challenges  and  Concerns. 

a.  Leader  Development.  Leader  Development  remains  our  #1  priority  and  is  essential  to 
developing  a  professional  ANSF.  Leadership  deficiencies  within  the  ANSF — across  the 
spectrum  from  insufficient  numbers  of  junior  officers  and  NCOs.  gaps  in  the  midgrade  ranks,  to 
corrupt  senior  officers — pose  the  greatest  threat  to  our  Afghan  allies.  Significant  efforts  have 
been  made  to  improve  leader  development  programs,  to  include  adding  new  and  overhauling 
existing  ANA  and  ANP  leader  courses.  However,  critical  shortages  in  officers  and 
noncommissioned  officers,  as  well  as  qualified  logisticians,  persist.  There  :s  a  need  lo  continue 
to  invest  energy  and  creativity  into  ANSI-  leader  development,  and  we  will  continue  to  focus  on 
junior  officers  and  noncommissioned  officer  programs  With  regard  to  midgrade  officers,  to  the 
degree  we  can.  we  will  continue  to  assist  our  Afghan  allies  in  the  field  to  encourage  the  selection 
of  officers  based  on  merit  and  not  cronyism.  To  help  address  senior  officer  leadership 
deficiencies,  our  ministerial  advisors  and  Hag  officers  will  work  with  our  Afghan  senior  partners 
to  develop  policies,  practices,  and  law  that  will  encourage  upward  mobility  and  the  selection  of 
senior  officers  based  on  merit.  Along  these  lines,  wc  will  support  the  passage  of  legislation  or 
policy  such  as  the  pending  Inherent  Law  and  related  retirement  programs 

b.  Attrition  Impacts  on  Quality  and  Quantity  As  we  executed  our  accelerated  growth 
plans,  it  became  apparent  that  a  complex  interaction  exists  amongst  recruiting,  retention,  and 
attrition.  This  interaction  alfects  our  efforts  to  meet  quantitative  goals  while  maintaining 
adequate  quality.  Even  if  the  attainment  of  recruiting  goals  and  expanded  training  capacity  allow 
us  to  meet  accelerated  growth  objectives,  attrition  must  be  brought  down  or  quality  will  suffer. 
Therefore,  one  of  our  major  challenges  :s  to  balance  there  various  demands.  A  natural  tension 
also  exists  between  today's  operational  requirements  at  one  end  of  the  spectrum,  and  alternative 
long-term  programs  focused  on  growth,  quality,  and  supporting  institutions  at  the  other  end.  An 
excessively  long  training  and  education  program  deprives  the  field  of  required  personnel .  We 
are  mindful  of  that  concern  and  modified  programs  accordingly.  For  example,  we  expanded 
recruit  ‘throughput'  by  modifying  the  daily  program  for  police  basic  training  (e  g.,  lengthening 
the  training  day  which  allowed  us  to  reduce  the  length  of  the  program  from  eight  to  six  weeks, 
and  thereby  increase  throughput).  The  number  of  professional  development  courses  for  army 
NCOs  has  also  been  condensed,  and  the  three  year  police  academy  is  being  converted,  for  the 
next  few  years,  to  a  six  month  OCS-typc  program.  On  the  other  end  of  the  spectrum,  the 
deployment  of  a  large  percentage  of  the  existing  ANSF  in  near  continuous  operations,  and  The 
challenge  of  rotating  combat  veterans  to  staff' training  institutions  or  manning  ETTs  in  a  timely 
fashion,  risk  retarding  long-term  professionalization  efforts.  Implementation  of  an  effective  and 
consistent  operariona  deployment  cycle  (rotation  between  train,  employ,  reset),  applied  to  both 
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the  ANA  and  ANP,  could  help  reconcile  the  competing  demands.  Such  a  cycle,  if  consistently 
observed  by  Afghan  rank  and  file,  and  augmented  by  a  robust  partnering  program  with  Coalition 
forces,  should  contribute  to  increased  retention,  reduced  attrition,  and  as  secondary  order  effects, 
raise  the  quality  of  the  force  while  meeting  growth  goals.  Lastly,  as  will  be  discussed  below,  in 
order  to  maximize  the  probability  of  meeting  both  quantitative  and  qualitative  objectives, 
training  and  instructor  shortfalls  must  be  filled  when  required. 

c.  Structure,  size,  and  nature  of  the  AN  A/ ANP.  To  build  an  appropriate  security 
framework,  and  to  produce  the  right  balance  and  type  of  security  forces,  to  include  sustainment 
capabilities,  NTM  must  remain  adaptable  We  must  also  closely  monitor  ethnic  balance  as  we 
grow  the  ANSF.  Arid,  wc  will  continue  to  assess  the  ultimate  size  of  the  ANSF  and  the  ratio 
between  army  and  police  that  best  supports  security  requirements  and  the  ability  to  transition  the 
ANSF  to  a  lead  security  responsibility. 

d.  Shortfalls  in  NTM-A'CSTC-A  Manning.  While  NTM-A/CSTC-A  has  received  a 
significant  influx  of  U.S.  personnel  since  November  2009.  there  remain  unfilled  requirements. 
Wc  continue  to  actively  engage  with  the  Joint  Start’,  the  Services,  and  SHAPE  to  bring  in 
personnel  with  the  appropriate  skill  sets  to  meet  our  quantitative  goals  in  a  timely  manner  while 
raising  quality.  If  wc  should  receive  all  confirmed  NATO  pledges,  wc  will  have  approximately 
750  additional  personnel,  bringing  us  to  just  over  80%  fill  for  the  first  time 

e  Rule  of  l  aw.  Internal  security  depends  on  the  three  pillars  of  judiciary,  corrections, 
and  policing.  While  we  are  directly  responsible  for  supporting  policing,  other  organizations  arc 
responsible  for  the  development  of  corrections  and  the  judiciary.  The  lagging  progress  of  the 
judiciary  is  a  serious  concern,  and  if  corrective  measures  arc  not  taken  soon  to  accelerate  the 
development  of  the  justice  system,  the  professionalization  of  the  Afghan  police  will  be  put  at 
risk. 

6.  Conclusion  In  our  first  six  months  we  made  significant  gams  as  we  changed  our  overall 
approach  to  training  and  created  a  sense  of  urgency  in  building  the  foundation  for  an  enduring 
and  professional  ANSF.  While  aggregate  growth  objectives  appear  on  track,  the  traditional 
summer  recruiting  lull  will  be  a  challenge.  In  addition,  the  greatest  threats  to  building  an  Afghan 
quality  force  with  adequate  quantity  are  the  lack  of  effective  leaders  and  the  persistent  high 
attrition  in  specific  components  of  the  ANSF.  Though  not  assured,  wc  can  increase  our 
probability  of  success  by  fully  resourcing  NTM-A'CSTC-A  and  carefully  balancing  current 
operational  reeds  against  those  of  longer  term  growth  arid  professionalization  programs. 


WILLIAM  B.  CALDWELL  IV 
Lieutenant  General.  USA 
Commanding 
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The  following  are  SIGAR's  comments  on  NTM-A/CSTC-A's  letter  dated  June  10,  2010: 

1.  NTM-A/CSTC-A  said  our  report  relied  on  information  that  was  out  of  date.  Our  audit  work  was 
conducted  from  October  2009  through  May  2010  and  relied  on  interviews  and  information 
collected  over  that  entire  period,  including  field  observations  of  18  ANSF  units  made  during 
February  and  March  2010  and  detailed  reviews  of  ANSF  assessment  documents  dated  through 
March  29,  2010. 

2.  NTM-A/CSTC-A  acknowledges  it  faced  numerous  problems  in  training  ANSF  units  as  recently  as 
six  months  ago.  However,  NTM-A/CSTC-A  said  it  has  made  much  progress  since  May  2010, 
when  our  audit  ended  and  believes  our  report  does  not  reflect  that  progress.  NTM-A/CSTC-A 
provided  its  May  29,  2010  report  entitled  NTM-A  180  Day  Internal  Assessment  and  Way 
Forward,  as  part  of  its  official  comments,  indicating  that  its  report  provides  more  current 
information  than  the  observations  and  information  we  developed  during  our  work.  Our  audit 
work  was  conducted  from  October  2009  through  May  2010,  and  our  draft  report  was  circulated 
for  agency  comment  on  May  27,  2010,  two  days  prior  to  the  issuance  of  the  NTM-A/CSTC-A 
report.  NTM-A/CSTC-A's  report  recounts  challenges  faced  in  training  and  fielding  ANSF  forces, 
accomplishments  achieved  during  NTM-A/CSTC-A's  first  six  months,  and  potential  concerns  for 
future  operations;  the  report  does  not  address  ANSF  capability  assessments.  We  have  included 
NTM-A/CSTC-A's  report  on  pages  37  through  41. 

3.  In  its  comments,  NTM-A/CSTC-A  observes  that  it,  along  with  DOD  and  IJC,  had  harbored 
concerns  regarding  the  efficacy  of  the  CM  rating  system  since  the  fall  of  2009.  As  such,  NTM- 
A/CSTC-A  cited  an  October  2009  DOD  report  to  Congress  entitled  Report  on  Progress  toward 
Security  and  Stability  in  Afghanistan.  That  report,  which  expressed  concern  about  the  CM  rating 
system  was  dated  October  2009,  but  was  not  issued  until  the  first  week  of  April  2010,  just  weeks 
after  we  shared  our  preliminary  observations  regarding  the  CM  assessment  at  an  out-brief  for 
the  Commanders  of  ISAF,  NTM-A/CSTC-A,  and  IJC.  The  DOD  report  passage  quoted  by  NTM- 
A/CSTC-A  contains  an  inaccurate  characterization  of  the  CM  rating  system,  stating  "CM  ratings 
simply  depict  the  manning,  training,  and  equipment  of  a  unit."  In  fact,  as  our  audit  report 
describes,  CM  rating  assessments  evaluated  a  variety  of  other  factors,  such  as  facilities, 
sustainment,  and  command  and  control.  NTM-A/CSTC-A  also  said  that  it  and  IJC  began 
reforming  the  CM  system  in  the  fall  of  2009.  While  we  appreciate  that  concerns  existed 
regarding  shortcomings  of  the  CM  system  during  that  timeframe,  IJC  and  NTM-A  were  only 
established  in  October  and  November  2009,  and  were  not  fully  operational  until  November 
2009  and  February  2010  respectively,  according  to  DOD's  April  28,  2010  report  to  Congress. 

4.  In  our  report,  we  discussed  construction  delays  and  resulting  conditions  at  the  Kunduz  Garrison, 
but  not  cost  issues.  NTM-A/CSTC-A's  comments  appear  to  address  a  prior  SIGAR  report,  ANA 
Garrison  at  Kunduz  Does  Not  Meet  All  Quality  and  Oversight  Reguirements;  Serious  Soil  Issues 
Need  to  Be  Addressed,  10-09,  April  20,  2010. 
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l  PACIFIC 
SOURmr  AtT  MRS 


OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 

2700  OerCNSC  PCNTAGON 
Washington,  DC  20301-2700 


Mr.  John  Rrummct 

Assistant  Inspector  General  for  Audits 
Office  of  the  Special  Inspector  General 
for  Afghanistan  Reconstruction 
400  Army-Navy  Drive 
Arlington  VA  22202 


Dour  Mr.  Brummet: 


The  following  is  the  Office  of  the  Secretary  of  Defense  (OSD)  comment  on  the 
Special  Inspector  General  for  Afghanistan  Reconstruction  (S1GAR)  draft  report.  “Actions 
Needed  to  Improve  the  Reliability  of  Afghan  Security  Force  Assessments,”  dated  June 
2010  (SIGAR  Audit-10-11). 

Page  13.  para  1:  "DOD  Reported  Capability  Ratings  for  Afghan  Police  Units  That 
Had  Not  Been  Assessed  by  Mentors  or  Partners”  (revised  from  "Reports  to  Congress  on 
Afghan  Police  Capabilities  Have  Been  Misleading”  in  report  draft). 


OSD  response:  OSD  appreciates  SIGAR’s  revisions  to  the  draft  report.  We 
remain  concerned,  however,  thut  die  language  still  does  not  provide  sufficient  clarity 
regarding  the  Afghan  National  Police  assessments  and  does  not  recognize  that  in  (lie 
October  2009  Report  on  Progress  Toward  Security'  and  Stability ’  in  Afghanistan ,  DOD 
pointed  out  the  w  eakness  of  the  Capability  Milestone  (CM)  system  as  an  important  risk 
factor  and  DOD’s  April  20 1 0  Report  on  Progress  Toward  Security  and  Stability  in 
Afghanistan  states,  “It  is  also  important  to  note  dial  the  current  CM  ratings  look  only  al 
the  manning,  training,  and  equipping  of  a  unit,  so  a  combat  unit  can  be  operationally 
effective  without  necessarily  being  rated  al  C.M1.”  By  nut  including  this  language  in  the 
SIGAR  audit,  the  audit  leaves  the  impression  that  DOD  had  not  recognized  this  risk 
problem  and  was  not  taking  steps  to  address  it  Therefore  to  report  accurately  on  DOD’s 
actions  on  CM  ratings,  we  recommend  that  the  section  title  and  first  paragraph  on  page 
13  of  the  draft  SIGAR  Audit-10-1 1  be  changed  to  read  as  follows: 

DOD  Reported  CM4  Capability  Ratings  for  Afghan  Police  Units  That  Had 
Not  Been  Assessed  by  Mentors  or  Partners 

Afghan  police  TRAT  summary  reports  we  reviewed  only  included  CM  ratings  for 
police  units  that  had  been  assessed  by  mentors  or  partners.  However,  in  some 
reports  to  Cotigress.  DOD  reported  CM4  capability  ratings  for  police  units  that 
CSTC-A  assessed  for  adequate  manning,  equipment  and  basic  training,  but  that 


SIGAR  Audit-10-11  Security/ANSF  Capability  Ratings 


Page  44 


had  not  actually  been  assessed  by  police  mentors  or  partners  for  operational 
effectiveness.  In  biannual  reports  to  Congress,  prior  to  April  2010,  Dclcnse 
reported  capability  ratings  for  as  many  as  559  ANP  units.  However,  as  of  March 
2010.  only  229  police  units  were  being  directly  mentored  or  partnered  and 
assessed  using  the  CM  system,  according  to  IJC  reports.  1JC  and  N  TM-A/CS1 C* A 
officials  agreed  that  the  data  used  to  create  the  charts  included  in  DOD's  June  2009 
and  October  2009  reports  to  Congress  on  Progress  toward  Security  and  Stability > 
in  Afghanistan  included  CM4  ratings  for  units  that  had  not  been  assessed  by  police 
mentors  or  partners.  Regarding  this  matter,  OSD  told  us  that  the  DOD  reports  had 
relied  on  data  from  CSTC-A  which  included  CM4  ratings  for  units  that  had  been 
provided  equipment  or  training,  but  had  not  been  assessed  by  mentors  or  partners. 
CSTC-A  had  provided  equipment  or  training  to  these  units  but,  due  to  force  level 
constraints,  partner  units  or  mentors  had  not  been  assigned.  According  to  IJC  and 
NTM-A/CSTC-A  officials,  a  continued  shortage  of  mentors  had  limited  police 
development  efforts  to  the  subset  of  police  districts  identified  as  most  important  to 
achieve  campaign  plan  goals  and  in  which  FPDD  or  In-District  Retonn  (IDR) 
training  had  been  provided  and  police  mentorvpartners  were  present.  DOD  s 
October  2009  Report  to  Congress  on  Progress  toward  Security  and  Stability -  in 
Afghanistan  stated  that  “the  utility  of  the  CM  rating  system  is  limited  to  measuring 
the  number  of  troops  trained,  equipped,  and  present  for  duly.  The  CM  system 
docs  not  effectively  measure  the  quality  of  performance  of  the  ANP.  This  is  a 
significant  risk  and  will  be  revisited  over  the  coming  year.  |  Add  footnote: 

DoD's  October  2009  Report  on  Progress  Toward  Security  and  Stability  in 
Afghanistan .  p.  30.1  Because  the  system  for  ANSF  assessment  was  under  review  , 
DOD  did  not  include  an  ANP  ratings  chart  in  its  April  28,  2010  report  to 
Congress.  DOD  had  recognized  the  shortfalls  of  the  CM  rating  system  to  reflect 
ANP  operational  capabilities  adequately  by  stating  in  the  report,  “It  is  also 
important  to  note  that  the  current  CM  ratings  look  only  at  the  manning,  training, 
and  equipping  of  a  unit,  so  a  combat  unit  can  be  operationally  effective  w  ithout 
necessarily  being  rated  at  CM I.”  [Add  footnote:  DoD’s  April  2010  Report  on 
Progress  Toward  Security  and  Stability  in  Afghanistan,  p.  111.] 

Thank  you  for  the  opportunity  to  comment  on  this  report.  My  points  of  contact  arc  Cara 

Negrettc.  703-695-8268.  Cara. Ncgrcttc</ .osd.mil  and  Mark  Morehouse,  703-695-8258. 

Mark.Morchousefdosd.mil. 


Afghanistan.  Pakistan  &  Central  Asia 
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APPENDIX  V:  COMMENTS  FROM  THE  OFFICE  OF  THE  SECRETARY  OF  DEFENSE  (PERSONNEL 
AND  READINESS) 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

4000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-4000 


PE»50NN*L  AND 

RCAOINCtt 

June  23,2010 

MEMORANDUM  FOR  OFFICE  OF  THE  SPECIAL  INSPECTOR  GENERAL  FOR 

AFGHANISTAN  RECONSTRUCTION  (SIGAR) 

(ATTN:  Ms.  Emily  Rachman,  Senior  Auditor) 

SUBJECT:  Draft  SIGAR  Report  -  Actions  Needed  to  Improve  the  Reliability  of  Afghan 
Security  Force  Assessments 


In  response  to  draft  SIGAR  Report,  Actions  Needed  to  Improve  the  Reliability  of 
Afghan  Security  Force  Assessments ,  Readiness  Programming  &  Assessment  identifies 
several  comments  of  significance.  Specific  comments  are  provided  in  the  attached 
comment  matrix,  SD  F orm  818.  OL  SD  (P&R),  RP&A  appreciates  the  opportunity  to 
provide  comments  on  the  report.  The  point  of  contact  for  this  action  is  Mr.  Michael 
Delacniz,  Senior  Readiness  Analyst  who  can  be  reached  at  (703)  693-3864  and 
michacl.delacruz  a  osd.mil. 


Brent  Barrow 
Deputy  Director, 

Readiness  Programming  &  Assessment 
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UNCLASSIFIED 


COMMENTS  MATRIX:  SECDEF  MEMO  -  Draft  SIGAR  Report:  Actions  Needed  to  Improve  .ANSF  Assessments 

tPlease  read  instructions  on  back  before  completing  form. ) 

it 

CUSS 

Component  and 
POCXame.  Phone, 
and  E-mail 

Page# 

Pm# 

Comment 
Type  (CVS) 

Comments.  Justific  ahon.  and  Originator  Jvstification  for  Resolution 

A' 

R' 

P 

P&RRPA 
Joseph  Angello 
693-5584 
losephangelloi; 


osd.mil 

Jim  Tolle 
703-604-0482 
liimes.tolle.ctrfto 

sd.mil 


Coordinator  Comment:  Add  the  following  background  information . 
where  appropriate,  before  the  last  paragraph  on  page  2: 

"From  the  end  of  2007  to  the  beginning  of  2009,  OSD  worked  with 
CENTCOM  and  CSTC-A  to  improve  ANSF  assessment  methodology  and 
reportmg.  CSTC-A  developed  an  enteipnse  reporting  capability  for  ETTs 
to  report  monthly  assessments,  including  capability-based  assessments 
and  CM  ratings.  A  web-based  version  of  TRAT  reporting,  called 
WeblRAT  was  deployed  throughout  the  theater  winch  supported  internet 
reportmg  of  ETT  and  OMLT  assessments  on  all  Corps  and  below  units. 
CSTC-A  worked  with  die  Afghan  Ministry  of  Defense  to  establish 
training  doctrine  which  defined  Mission  Essential  Tasks  (METs)  for  ANA 
units  and  these  tasks  were  used  by  the  ETTs  to  report  monthly 
assessments  in  accordance  with  U.  S.  Army  assessment  doctrine.  CSTC- 
A  OPORDER  dated  08  May  2008  directed  die  reportmg  of  monthly  MET 
assessments  and  CM  ratings  into  die  Department  in  the  Defense 
Readiness  Reportmg  System  m  order  to  support  more  streamlined 
reportmg  and  sharing  of  assessments  leadership  m  theater  and  throughout 
die  Department  for  national  analysis.  A  similar  reportmg  capability  was 
attempted  for  ANP  with  PMT  mentors  reportmg  m  an  enteipnse  system 
from  August  to  December  2008  but  many  problems  similar  to  those 
covered  m  die  remainder  of  the  report  prevented  reliable  ANP  assessment 
reportmg.'' 

Rationale:  Document  previous  substantive  efforts  by  CSTC-A. 
CENTCOM  and  OSD  to  address  reliability  and  integrity  problems  with 
die  CM  ratmg  system  and  to  perform  ETT  assessments  of  ANA  forces 
using  a  MET-based  methodology  similar  to  how  the  US  assesses  its  own 
fighting  units. 


SD  FORM  818.  JAN  09 
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UNCLASSIFIED 


COMMENTS  MATRIX:  SECDEF  MEMO  -  Draft  SIGAR  Report:  Actions  Needed  to  Improve  ANSF  Assessments 

(Please  read  instructions  on  back  before  completing  form.) 


(USS 

Component  and 
POC  Name,  Phone, 
axdE-mail 

Page# 

Para# 

Comment 
Type (CVS) 

Comments.  Justification,  .and  Originator  Justification  for  Resolution 

2 

U 

P&RRPA 
Joseph  Aneello 
693  -5)84 
iosephangello® 

osd.mil 

JimTolle 

703-604-0482 

iames.tolle.ctr'So 

sd.mil 

6 

2 

S 

Recommendation:  Add  the  following  paragraph  as  appropriate  to  the 
section  on  "The  Measurement  and  Validation  of  ANSF  Capabilities  May 
Have  Resulted  in  Overstatement  and  Variability  in  Assessment  Results  ’: 

"OSD  and  CSTC-A  efforts  from  2007  to  2009  to  improve  the  ETT 
assessments  using  an  enteipnse  MET-based  reporting  system  were  not 
sustamed  by  forces  m  theater  despite  efforts  to  establish  theater-wide 
policy,  tranung  and  support.  Inconsistent  implementation  of  task  force 
policy  and  rotation  of  personnel  led  to  enterprise  and  MET-based 
reporting  through  WebTRAT  being  replaced  by  the  Excel-based 

Capability  Endorsement  system  in  2009.  Inconsistencies  in  implementing 
assessment  methodology  improvements  and  sustaining  consistent 
reporting  systems  over  tune  may  have  contributed  substantially  to  loss  of 
data  and  variability  of  the  assessment  data  m  2009. 

Rationale:  To  document  previous  substantive  efforts  by  CSTC-A 
CENTC'OM  and  OSD  to  address  reliability  and  integrity  problems  with 
the  CM  rating  system  and  how  problems  in  sustaining  task  force  reporting 
policy  and  systems  contributed  to  variability  in  reporting  and  assessments. 

3 

U 

P&RRPA 
Joseph  Angello 
693  -5584 
i  oseph.  angelloi® 

osd.mil 

Jun  Tolle 
703-604-0482 
iames.tolle.ctr@o 

sd.mil 

9 

2 

S 

Recommendation:  Add  at  the  beginning  of  the  section  "Weaknesses  in 
ANSF  Evaluation  Tools  Yielded  Inconsistent  Msessment  Results 
"Efforts  in  2007  to  2009  to  establish  an  standardized,  enteipnse  reporting 
tool  through  WebTRAT  and  DRRS  for  the  MET-based  assessment  of 

ANA  forces  by  ETTs  and  OMLTs  were  not  sustamed  over  time, 
inconsistency  in  assessment  procedures  and  methodologies  have  affected 
ANA  assessments  since  then. 

Rationale:  Document  how  efforts  by  CSTC-A.  CENTCOM  and  OSD  to 
address  reliability  and  integrity  problems  with  the  CM  rating  system  were 

SD  FORM  818.  JAN  09  PREITOIS  EDITION  IS  OBSOLETE 
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UNCLASSIFIED 

COMMENTS  TUTRIX:  SECDEF  MEMO  -  Draft  SIGAR  Report:  Actions  Needed  to  Improve  AN’SF  Assessments 

(Please  read  instructions  on  bad  before  completing  form.) 

# 

Cuss 

Component  and 
POC  Name.  Phone. 
AND  E-MAIL 

Pages 

Para* 

Comment 
Type  (C/S) 

Comments,  Justification,  .and  Originator  Justification  tor  Resolution 

A/ 

R/ 

P 

not  sustained  and  how  problems  m  sustaining  task  force  reporting  policy 
and  systems  contributed  to  problems  in  consistency  and  reliability  of 
assessments  following  this  period. 

4 

U 

P&RRPA 
Joseph  AngeDo 
693  -5584 
ioseph.angello  d 

osd.mil 

JimTolle 

703-604-0482 

iames.tolle.ctrfSo 

sd.mil 

19 

1 

S 

Recommendation:  Change  first  recommendation  to  UC  to  read  - 

"Place  greater  emphasis  on  subjective  assessment  of  overall  operational 
capability,  based  on  Mission  Essential  Tasks  ( METs j  where  appropriate, 
provided  by  mentor  and  partner  team  evaluators,  including  a  requirement 
to  complete  narrative  observations." 

Rationale:  To  improve  the  credibility  and  reliability  of  the  ANSF 
assessments.  ISAF  assessments  should  reflect  the  same  MET-based 
methodology  and  approach  as  used  by  U.  $.  forces  in  accordance  with 
DODD  7700.65  and  AR  220-1. 

(This  report  was  conducted  under  the  audit  project  code  SIGAR-012A.) 
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SIGAR’s  Mission 

The  mission  of  the  Special  Inspector  General  for  Afghanistan 
Reconstruction  is  to  enhance  oversight  of  programs  for  the 
reconstruction  of  Afghanistan  by  conducting  independent  and 
objective  audits,  inspections,  and  investigations  on  the  use  of 
taxpayer  dollars  and  related  funds.  SIGAR  works  to  provide 
accurate  and  balanced  information,  evaluations,  analysis,  and 
recommendations  to  help  the  U.S.  Congress,  U.S.  agencies,  and 
other  decision-makers  to  make  informed  oversight,  policy,  and 
funding  decisions  to: 

•  improve  effectiveness  of  the  overall  reconstruction  strategy 
and  its  component  programs; 

•  improve  management  and  accountability  over  funds 
administered  by  U.S.  and  Afghan  agencies  and  their 
contractors; 

•  improve  contracting  and  contract  management  processes; 

•  prevent  fraud,  waste,  and  abuse;  and 

•  advance  U.S.  interests  in  reconstructing  Afghanistan. 

Obtaining  Copies  of  SIGAR 
Reports  and  Testimonies 

To  obtain  copies  of  SIGAR  documents  at  no  cost,  go  to 
SIGAR’s  Web  site  (www.sigar.mil).  SIGAR  posts  all  released 
reports,  testimonies,  and  correspondence  on  its  Web  site. 

To  Report  Fraud,  Waste,  and 
Abuse  in  Afghanistan 
Reconstruction  Programs 

To  help  prevent  fraud,  waste,  and  abuse  by  reporting 
allegations  of  fraud,  waste,  abuse,  mismanagement,  and 
reprisal  contact  SIGAR’s  hotline: 

•  Web:  www.sigar.mil/fraud 

•  Email:  hotline@sigar.mil 

•  Phone  Afghanistan:  +93  (0)  700-10-7300 

•  Phone  DSN  Afghanistan  3 1 8-237-2575 

•  Phone  International:  +1-866-329-8893 

•  Phone  DSN  International:  312-664-0378 

•  U.S.  fax:  +1-703-604-0983 

Public  Affairs 

Public  Affairs  Officer 

•  Phone:  703-602-8742 

•  Email:  PublicAffairs@sigar.mil 

•  Mail:  SIGAR  Public  Affairs 

400  Army  Navy  Drive 

Arlington,  VA  22202 

